dr|z3d
mockup of missing inputs in the i2ptunnel server throttle: cake.i2p/file/lQZMfqYVw9_0HYfXno3A4Ne0ddHCO9x4xgQJ23u9W_YpI4uOLzW2/throttle.webp
dr|z3d
new is the Ban duration configs per minute/hour/day for requests (per client / total), and the limit on concurrent requests per client.
dr|z3d
fao zzz, eyedeekay
dr|z3d
ban duration could equally be the trailing column, though table might get more cramped that way.
anonymousmaybe
Setting up i2p-keyring (2021.09.10) ...
anonymousmaybe
Adding/refreshing keys:
anonymousmaybe
67ECE5605BCF1346 1381446626 [ expires 1696084436 ]
anonymousmaybe
Blinded message
anonymousmaybe
nerscript of the package i2p-keyring)
anonymousmaybe
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (s
anonymousmaybe
ee apt-key(8)).
anonymousmaybe
i think since the instructions changed for I2P on how to add the keys using the new method
anonymousmaybe
then installation of I2P should do the same
anonymousmaybe
cc eyedeekay zzz
zzz
dr|z3d, thats not how it works, it's bucketed
eyedeekay
anonymousmaybe there is a series of conditions that are expressed in the installed in i2p.keyring MR #1 i2pgit.org/i2p-hackers/i2p.keyring.i2p/-/merge_requests/1
eyedeekay
whether apt-key is used or whether a specific key is placed in the sources.list file depends on the version of Debian in use
eyedeekay
I've tested it but it would probably get in faster if third-parties tested it too
eyedeekay
it is the MR that does the thing you're asking about
zlatinb
Interesting thought/question - can the I2NP ports not be opened at all if the router is forced to behave as if firewalled? github.com/zlatinb/muwire/issues/143
dr|z3d
zzz: which part of that mockup isn't viable? or is there another way to implement the proposed additional features?
zzz
zlatinb, why?
zzz
dr|z3d, a UI is a configuration interface to underlying code that implements what you want
zzz
that code doesn't exist because the counting/banning is done in buckets
zzz
perhaps access filter will do what you want, take a look at it
dr|z3d
no, access filter doesn't support setting limits on concurrency.
zzz
I was talking about ban time
zzz
you are correct, there's no concurrency limit code anywhere
dr|z3d
apparently there is, otherwise how can you set a total limit for all clients?
zzz
oh yeah, forgot about that
dr|z3d
and I don't think access filter supports setting a ban duration, either.
zzz
I think you're wrong about that
dr|z3d
the access filter allows you to set the max number of requests per period, not the same thing.
dr|z3d
if you add a router to your permanent deny list, it stays there, there's no configurable duration after which it's removed.
zzz
zlatinb would be the one to ask
dr|z3d
you hit that issue a while back with the 2K+ b32s in your deny list.
dr|z3d
zlatinb wrote some documentation on the feature a while back, there's no mention of ban duration.
zzz
i2p-projekt.i2p/spec/filter-format yeah I don't see it
zzz
oh well
dr|z3d
you can currently set a duration for post limit bans, and that's it, for the throttler. so it leads me to believe the throttler is incomplete.
zzz
the streaming throttler is just buckets
dr|z3d
which means what exactly? how does that prevent setting a period for bans for requests, but not for post requests?
zzz
it's two completely different implementations
dr|z3d
well there _is_ a ban duration for requests, it's just not exposed in the UI.
zzz
buckets means the ban time is the remainder of the bucket time. The bucket (the counts and the bans) is cleared at the end of the interval
zzz
agreed there is a ban time. But it's neither constant nor configurable
dr|z3d
so the magic word here is interval.. it's hard-coded in the backend, just not exposed in the UI for requests, post-requests excepted.
zzz
it's hardcoded but not like you think
zzz
I think you still don't get it? should I try again?
dr|z3d
sure, humor me :)
zzz
it's 3:44 pm
zzz
I've been counting in my hour bucket that started at 3 PM
zzz
once you exceed the limit, you're banned
zzz
I reset all bans and counts to zero at 4 PM, and everybody is unbanned
zzz
so the ban time and the interval time are not independent
zzz
got it?
dr|z3d
I get that for the total request count, I don't get that for the per-client count. Why would the per-client count be bucketed with all clients?
zzz
simplicity and memory usage
zzz
there's no timestamps stored at all
dr|z3d
so there's a period.. say I trigger a throttle at 1 minute before the end of the period.. my ban persists for 1 minute?
zzz
yup
dr|z3d
that's not great.
zzz
nope, it isn't, it's just something basic
dr|z3d
there's no predictability for the visitor. "oh, I've triggered a block, could be for 1 minute, could be for 9 minutes".
zzz
I posit that predictability is bad. there's no obligation or value in that
dr|z3d
sure, notwithstanding the above, there's nothing that stops the bucket period from being configurable in the UI, right?
zzz
but we do send a retry-after header that's not a total lie, iirc
zzz
could be configurable, but we offer three buckets already, might be enough
dr|z3d
ok, I think I'm getting it.. so the period determines the max ban time as it stands.
zzz
yup
dr|z3d
if I trigger a block within the per day limits, I'm blocked for potentially up to a day?
zzz
yup
dr|z3d
that's horrendous!
zzz
depends if the person you're blocking is a good guy or a bad guy
dr|z3d
still, I'm better informed as to what it's doing now, so thanks for the explanation.
zzz
but yes, set per-day limits with extreme caution
zzz
access filter does do timestamps, not buckets, so it is a lttle smarter
dr|z3d
that's true, but you've know way of knowing what's good or bad with the current implementation. incremental blocking period could be implemented if we kept track of individual dests.
zzz
some external reverse proxy solution is probably more suited to your requirements
dr|z3d
yeah, except as we've discussed, access filters don't allow you to configure a ban period, it's another fairly blunt tool.
zzz
sure, it's only code and CPU and memory, anything could be more complex
zzz
you may wish to write up a proposal for access filter enhancement and run it by zlatinb
zzz
might also be straightforward to write your own access filter, drop it in a jar, then somehow we provide a config to use it
zzz
*relatively straightforward
zzz
also FYI I dropped a preliminary test plan over on my forum
dr|z3d
yeah, saw that.
dr|z3d
reponse "it all works".
dr|z3d
:)
dr|z3d
some of the features won't be happening, not viable.
zzz
worked once or works every time? :)
dr|z3d
works reliably.
zzz
great
dr|z3d
auth for higher limits won't be happening.
dr|z3d
no way to implement that, and creating a tiered service is a bad idea in any event in my opinion.
dr|z3d
we could require auth for all users, though. :)
zzz
don't read optional section as a wishlist
zzz
just things to be tested if they are offered
zzz
and open to comments moving features to/from optional, just a first cut
zzz
those with more Tor knowledge than I may have better insights on best practices
dr|z3d
Tor isn't an issue. It either works or it doesn't. Push all .onion requests and anything else you don't want direct access to over a Tor instance. done.
zzz
I'm talking about what tor does with headers, DNS, etc. at its exit nodes may help with recommendations in those areas
dr|z3d
headers are filtered before they hit Tor. not a problem.
zzz
specifically adding proxy headers
dr|z3d
we don't/won't do that. Via headers, no.
dr|z3d
to all intents and purposes, a request should appear native, not forwarded via a proxy.
zzz
but does tor? It might help reduce throttling or abuse reports from end servers, perhaps
zzz
at some tradeoff with user privacy
zzz
that's the stuff I don't have good answers for
anonymousmaybe
eyedeekay im installing I2P in debian bullseye, But during the installation something is messed in i2p-keyring you might try to check
anonymousmaybe
its reproducible in every test, easy to duplicate the same warning message at your end
dr|z3d
I'm fairly confident Tor doesn't mess with headers, but let me just confirm.
zzz
anonymousmaybe, you've reported this many many many times in the past year and submitted several tickets about it, including duplicates, we really don't need to hear more about it
anonymousmaybe
zzz i was reporting about the instructions = instructions fixed = good
anonymousmaybe
now during installations, didnt open a ticket about afaik
zzz
you're reporting about the warning. we know about the warning. there's multiple tickets and a pending MR already
dr|z3d
what might be an idea from i2ptunnel's perspective is to add the TorBrowser useragent header when .onion sites are requested over the proxy tunnel.
zzz
I know I've closed duplicate tickets from you about it
anonymousmaybe
ah ok great then, good to know
zzz
if the warning becomes an error, then let us know
zzz
anonymousmaybe, git.idk.i2p/i2p-hackers/i2p.i2p/-/issues/352
dr|z3d
oh, actually, strike that, should already be happening I guess.
dr|z3d
and no, Tor apparently doesn't mess with headers at all.
dr|z3d
which isn't surprising, given their overall stance on modifying requests in flight.
anonymousmaybe
zzz damn im getting old, i totally forgot this report
zzz
i just don't know if a proxy would get more lenience, or less, or don't care, from e.g. twitter if a Via or X-Forwarded header was set
zzz
I suppose VPNs probably don't declare themselves
dr|z3d
in this particular instance, pointless, even if it was desirable (which it's not). ip range is already flagged as running Tor, so will get treated accordingly.
dr|z3d
work on the basic assumption that the net doesn't like proxied connections. then you're in the right ball park. so we don't want to flag that. :)
zzz
that's why its not a wishlist, just things to make an actual decision on based on the goals and best practices
StormyCloud
I would think we want traffic to appear as coming from a "normal" user. Not flagging the traffic as a proxy
zzz
ohai stormyC
StormyCloud
Howdy
zzz
discussing bottom of zzz.i2p/topics/3254
StormyCloud
Yup been reading and listening to yall discuss
zzz
dr|z3d, there are hop-by-hop headers that I would think even tor would need to strip, but idk
zzz
e.g. Proxy-Connection must be stripped by any outproxy, I'm pretty sure
zzz
and would be a giveaway if not
dr|z3d
possibly, zzz. the important point is that Tor doesn't add headers that flag a request as explicitly coming from Tor.
zzz
sure, but if you don't know about all the hop-by-hop headers and aren't stripping them it's very wrong and revealing something
zzz
thats why this isn't so simple
zzz
this is also a case where the java http server tunnel does you a lot of favors that i2pd probably doesnt
zzz
and a std. tunnel definitely doesn't
dr|z3d
we whitelist headers, and drop everything else.
zzz
i2pd just fixed connection: close a couple days ago
dr|z3d
i2pd might appear to be an obvious choice for hosting an outproxy, but as you say, it's missing a bunch of fine-grained controls and management/logging tools that make it less than optimal vs Java.
dr|z3d
anyways, purokishi's been running fine for the last 2 years or more, no reason to suspect stormycloud.i2p will behave any differently.
zzz
I'm just talking basic RFC correctness. Not that we're some gold standard either.
zzz
sure, but we've never make a list of requirements or tests before, for yours or any other
zzz
I'm still not sure we do the CONNECT stuff right. client-side we return a 200 right away, RFCs say we're supposed to wait until we get a socket all the way thru
zzz
not sure thats possible though
dr|z3d
not sure, but what I can tell you is we can't send 404's via the outproxy for CONNECT. browser handles that.
dr|z3d
no problem.
zzz
yeah maybe in theory the server side proxy should be sending the 200/404?
dr|z3d
(with the notable exception of non-resolvable .onion addresses, as we've discussed.)
zzz
as it is now any server side response ends up after the client side 200
zzz
so we may have biffed that
dr|z3d
maybe a good time to revisit and consult relevant RFCs :)
zzz
the complication is that the client side and the server side act together as a single proxy, sortof, kinda
zzz
doesn't map neatly onto RFCs
zzz
and wasn't designed top-down or documented
zzz
with magic like x-i2p-gzip in the middle
zzz
and then mix and match java and i2pd on the two ends, or even a socks tunnel on the client side, and it gets real messy
dr|z3d
well, if you want to test your own proxy stack to make things easier, hit me up over pm.
zzz
pfft