+R4SAS
+RN
+RN_
+T3s|4
+Xeha
+acetone
+orignal
Irc2PGuest33667
Irc2PGuest97218
Onn4l7h
aargh2
b4dab00m
cumlord
eyedeekay_
leopold
not_bob_afk
profetik1
shiver_
u5657
vvx
x74a6
dr|z3d
zzz: re buckets, a separate but related proposal. hard limit for U peers @ 10% of total netdb.
zzz
thats hard but my recent changes to netdb HFDSMJ do pretty much the same thing when over limits
zzz
pls review those changes w.r.t. "unsolicited" stores
dr|z3d
I usually see < 10% until around 4K peers, then it ramps, and beyond around 5-6K peers, significant majority of new peers are U.
dr|z3d
you pushed something lately?
dr|z3d
let me have a look..
zzz
few days ago
dr|z3d
ah, then the code's already running all over.
zzz
those changes are doing a good job at keeping a lid on things
dr|z3d
great. I have every faith. you mean git.idk.i2p/i2p-hackers/i2p.i2p/-/commit/7a75ea4bef68cdc7deeefb4eb4725a8cd88e7786 ?
dr|z3d
> NetDB: Store handler updates
zzz
over the last two weeks I'm down from ~10k to ~5k RIs on my ff
zzz
sounds right
dr|z3d
5K is sane, 10K, not so much.
zzz
basically try to classify netdb store as "unsolicited" or not, drop the former aggressively if necessary
dr|z3d
I thought we already recognized unsolicited stores?
zzz
LS not RI
dr|z3d
ah.
dr|z3d
recent issue then?
zzz
you may have diverged so far from me and dropping so much that you're not seeing any effect from my changes
zzz
I'm still not sure if 'netdb spam' was from directly-connecting peers (aka solicited) or some sort of netdb mass store spam
dr|z3d
it's possible. but I'm not seeing any negative effects of said change, so carry on. :)
zzz
it's helping but not really sure whats going on, then or now
dr|z3d
yeah, I never saw a huge hike in LS stores, but maybe I wasn't paying attention.
zzz
speaking of divergence
zzz
want a pointer to about 8 bugs in 3 lines?
dr|z3d
always happy to receive your feeback as you know, especially when it's packaged so nicely, bow and all :)
zzz
I'll refrain from negativity other than to say do better...
zzz
PersistentDataStore.write()
zzz
RouterInfo ri = _context.netDb().lookupRouterInfoLocally(key);
zzz
boolean isHidden = _context.router().isHidden();
zzz
RouterInfo us = _context.netDb().lookupRouterInfoLocally(_context.routerHash());
zzz
boolean isUs = us != null && us.equals(_context.routerHash());
zzz
1) don't lookup your own RI in the netdb. do ctx.router().getRouterInfo()
zzz
2) you're looking to compare if the thing being stored is your router
zzz
3) the RI in question being stored is passed to you, i.e. databaseentry data
zzz
4) why are you looking it up again by key
zzz
5) if you want to see if it is yourself, just compare hashes
zzz
i.e. key.equals(_context.routerHash())
zzz
6) us.equals(_context.routerHash()) you are comparing a RouterInfo to a Hash. That will always return false. The compiler does not save you with equals() that are impossible
zzz
all in all just really really not right
zzz
EOT
dr|z3d
thanks
zzz
Cat cat; Dog dog; cat.equals(dog) will always return false, compiler won't tell you, you have to run findbugs to catch that stuff
dr|z3d
ok, I'll be mindful of not assuming cat equals dog (or not) in future.
zzz
yes it's a waste of 1KB of disk space to save your own RI to disk but your code never fires because of all the bugs, suggest you spend your time on more important things and/or test more
zzz
yeah impossible equals() is a trap, very hard to see or find those
zzz
but never lookup your own RI in the netdb and don't lookup anything if you have it already
dr|z3d
the intention of that code is not to keep my own RI off the disk, it's to keep all the crap RIs off the disk.
dr|z3d
ok, noted, thanks.
zzz
just compare hashes not RIs
zzz
and definitely not one to the other :)
dr|z3d
got it :)
zzz
if I wasn't clear, also, the DatabaseEntry passed in _is_ the RI being stored
zzz
dont go back and forth RI to hash to RI to hash. pick one or the other. don't get RI unless you need it, which you don't for basic isUs decisions
dr|z3d
so:
dr|z3d
RouterInfo ri = _context.netDb().lookupRouterInfoLocally(key);
dr|z3d
boolean isHidden = _context.router().isHidden();
dr|z3d
RouterInfo us = _context.router().getRouterInfo();
dr|z3d
boolean isUs = us != null && key.equals(_context.routerHash());
dr|z3d
ok, I guess it's pretty much irrelevant there anyways. write our own RI to disk or not, doesn't matter.
dr|z3d
and I can lose the RouterInfo us altogether is what you're saying I think.
dr|z3d
RouterInfo ri.. ok, getting there :)
dr|z3d
so really all I need there is:
dr|z3d
boolean isHidden = _context.router().isHidden();
dr|z3d
boolean isUs = key.equals(_context.routerHash());
dr|z3d
the rest is cruft.
dr|z3d
maybe I do need: RouterInfo ri = _context.netDb().lookupRouterInfoLocally(key); for the version stuff, but yeah.
zzz
right
zzz
even if you did want to check, all you need was key.equals(ctx.routerHash())\
zzz
no lookups reqd
dr|z3d
so just 3 lines. I like it, much cleaner. thanks for the guidance.
dr|z3d
RouterInfo ri = _context.netDb().lookupRouterInfoLocally(key);
zzz
no no dont do lookups
dr|z3d
boolean isHidden = _context.router().isHidden();
zzz
you don't need RIs
dr|z3d
boolean isUs = key.equals(_context.routerHash());
dr|z3d
yeah, I don't know why I was looking up my own RI, must have copied code from somewhere in error.
zzz
and even if you did, the RI for the key was passed in as databaseentry
zzz
hash comparison is sufficient
zzz
don't go back and forth hash/ri/hash
dr|z3d
ok, I can extract RI info from the hash without a lookup?
zzz
you don't need RI to know if it is you
dr|z3d
that's not the main point of that code.
zzz
if you want to look at caps, then yes you need it
zzz
so yeah
dr|z3d
if (ri != null) {
dr|z3d
v = ri.getVersion();
dr|z3d
unreachable = ri.getCapabilities().indexOf(Router.CAPABILITY_UNREACHABLE) >= 0;
dr|z3d
caps = ri.getCapabilities();
zzz
but just do RouterInfo ri = (RouterInfo) data
zzz
the ri is passed in
orignal
DtQs: 180.190.85.189:31157 => [1499:1066]
orignal
seem no problem with it
zzz
up to you orignal but it's on a different IP every minute
orignal
DtQs: 50.159.167.46:41932 => [1488:0]
orignal
yes
orignal
but what's with it?
dr|z3d
it's daf, orignal.
orignal
what is daf?
dr|z3d
dodgy as fsck.
orignal
who cares?
dr|z3d
the network cares.
dr|z3d
especially when it's requesting a huge number of tunnels.
orignal
I mean if they change IP every minute
dr|z3d
up to no good.
orignal
they don't publish any IP
dr|z3d
thanks, zzz, (RouterInfo) data; compiles just fine.
dr|z3d
ip changes every minute wouldn't be a problem of itself, it's the huge number of requests that are problematic.
dr|z3d
the two together indicate either a poorly configured router/vpn combo, or malicious intent. either way, undesirable on the network.
orignal
but since they don't publish thier IP they are allowed to do it
dr|z3d
not if you blacklist the hash.
orignal
and they don;t publish instroducers for SSU2
orignal
most likely they use proxy
dr|z3d
they're using a vpn no doubt.
zzz
there are really bad routers out there. this is an example. By not banning it you hurt the network. This is an example of why you need a ban list
orignal
router.version=^F0.9.55
zzz
but I guess we still haven't convinced you. Oh well. I tried.
orignal
but you did't provide any reason why should I ban him
orignal
changing IP is not enough
dr|z3d
a HUGE number of tunnel requests in a very short timeframe.
orignal
that's another question
dr|z3d
keep an eye on him, you'll see orignal..
zzz
I'll try again some other day
orignal
so the problem is not change IPs but tunnel requests
dr|z3d
don't give up, zzz, we're getting somewhere, I can feel it :)
dr|z3d
yes, orignal
dr|z3d
it's the amount of resources he's asking from the network.
orignal
huge number of requests, yes, it makes sense
dr|z3d
more tolerable from an XR class router that's also serving a bunch of tunnels, not so much from an unreachable (L?) class router about as useful to the network as a marzipan dildo..
zzz
change IP every minute is enough reason by itself
orignal
why?
zzz
you would never be able to connect to it
zzz
even with introducers
orignal
might be just a VPN
zzz
it's a terrible VPN then
dr|z3d
it _is_ a VPN.
zzz
if you can't keep the same IP for one minute you are totally f'ed
orignal
zzz, how do you think i2pd build inbound tunnels through Tor?
dr|z3d
probably the VPN has been explicitly configured to rotate IPs every minute, which leads one to question the motives of the router's operator.
orignal
so what's the theshold? 1 IP per minute?
zzz
the threshold is we identified this router is causing a lot of problems and is totally braindamaged so we banned it manually with our ban list
zzz
if you have a ban list then it is possible
orignal
then tell me the threshold for IP change
dr|z3d
which raises an interesting question..
dr|z3d
can we track ip changes per RI and start auto-banning if they reach a certain threshold per period, zzz?
dr|z3d
orignal's looking for some automated response, not a bad idea.
dr|z3d
(for everyone)
dr|z3d
5 ips per hour, zzz? what do we think?
zzz
this is not about automated thresholds.
zzz
it's about making a decision to manually ban very bad routers
zzz
if this example didn't convince orignal I will provide some more some later time
orignal
I prefer automatic
dr|z3d
yeah, that's what I thought.
dr|z3d
if orignal wants to automate banning bad routers, then he needs a threshold, which, if it snags our bad router and more, is totally fine. and probably worth considering for java.
orignal
more than 1 IP change in 5 minutes ban for 1 hour
dr|z3d
if we work on the basis that a leaseset is 10m, then 5 per hour seems like about the max you'd want for a semi-stable router, no?
dr|z3d
so more than 1 ip change every 12m = ban perhaps?
orignal
for how long?
dr|z3d
depends how often you want to keep banning. 4h seems about right?
orignal
agree
zzz
we do manual permanent banning by analysis, not a rule book. If you want to use your own rule book, fine
zzz
automated banning is different
zzz
DtQs was banned manually after analysis and discussion
zzz
our automated permanent/temporary banning is completely different
zzz
no we don't have automated banning for IP changes
orignal
then we should
dr|z3d
seconded.
zzz
I brought it up as an example of why a ban list is good and necessary. Argue about the rules some other day
dr|z3d
not disagreeing about a banlist, it's a handy tool of last resort, but does require manual intervention. otoh, a rules based system that excludes routers we can't connect to, as you said, is also a good idea, especially if it catches our badly behaved router and others like it.
dr|z3d
orignal: what about an opt in for the existing java banlist?
orignal
manual banlist takes YOUR time
dr|z3d
(in i2pd)
orignal
maybe
orignal
need to write the code first
dr|z3d
yup, that's what I'm getting at, it's a manual intervention thing.
zzz
this is item 3) from the recommendations. Identify bad peers and don't propagate their info through the network
zzz
manual vs. auto is an irrelevant detail right now
zzz
stop bad peers
dr|z3d
subscribing to a global banlist in i2pd does 2 things. 1) removes the requirement to vet all ips separately from java and 2) gives you input into what makes it on the list.
zzz
you are damaging the network if you don't
orignal
so I prefer automatic
dr|z3d
ips/hashes whatever.
dr|z3d
for the existing banlist, you'd just need to pull in the xml, orignal. you'd probably also want to implement a manual banlist where the user can add their own bans.
dr|z3d
for automated bans, you'd probably want to maintain a list that writes to disk so it's session-persistent.
dr|z3d
which is what java i2p does for sybil bans.
zzz
if i2pd wants to trust java i2p manual banning decisions, you can use our blocklist feed. But it doesn't sound like you do trust our decisions. that's fine.
dr|z3d
he's not dead set against, zzz: <orignal> need to write the code first
dr|z3d
using java's list also gets him a ticket to the party. so there's that :)
orignal
huh?
orignal
ofc I'm not going to receive your feed
orignal
but can process your su3 manually
dr|z3d
sure, pull the feed on a server somewhere, tweak as required, then serve as your own feed.
orignal
do you have an example of your file?
zzz
I get it, I'm sure you don't want me controlling your network ))
orignal
zzz, it's not about control
zzz
not offended ))
orignal
even if we had such feed it will come from our source
zzz
or roll your own
orignal
why xml not json?
zzz
the dr|z3d link above is a subset of the feed
zzz
it is what it is
orignal
for me it can be just flat list
zzz
if you want to parse ours, that's the spec, use it or don't
zzz
ok I got the probabalistic inbound conn throttler working
zzz
fine tuning it now
zzz
it has to be really tight to be effective
zzz
so tight I also had to write an "exemption" system so inbound tunnel builds can bypass it
zzz
02-16 14:26:46.399 WARN [NTCP Pumper ] ter.transport.ntcp.EventPumper: Probabalistically dropping incoming (p=36/128 last rate 28/min current rate 31
dr|z3d
nice, nice
zzz
goal is to not slam to 1K conns 2 minutes after startup
dr|z3d
sure, incline should be gradual, not ridiculous.
dr|z3d
are you testing it on something that sees that amount of traffic post-startup?
zzz
esp. ffs get hammered
zzz
taken about 25 restarts to fix it and dial it in but I'm getting close
dr|z3d
good. I have every confidence you'll have something you're happy with when you're finished.
zzz
as usual, started testing on a slow router, moving to faster routers as it gets better
dr|z3d
ok
dr|z3d
are we going to graph/stat the drops?
zzz
what are your tcp/udp conn limits on your biggest router (assuming not set manually) ?
dr|z3d
I think I set them manually, but let me look at the code, see what the defaults are.
zzz
or have you tweaked the defaults too?
dr|z3d
ofc. :)
zzz
((
zzz
then what are typ. tcp/udp conn counts on biggest router?
dr|z3d
I don't know what counts as the biggest router right now, but I'm looking at ~1600 NTCP, ~400 SSU2 on the console I've got open.
dr|z3d
usually more, let me find another to look at.
zzz
I think I want to tune it to get to full or typ. capacity in 15 minutes or so
dr|z3d
about the same on another (usually) high bw router.
zzz
ok thx
zzz
nothing crazy
dr|z3d
both those routers doing much less b/w than usual, so maybe a quiet day in paradise.
dr|z3d
maybe 15, 3, 5 are approx targets to hit in 15m, for ntcp/ssu1,ssu2.
dr|z3d
somewhere in that ballpark, anyways.
zzz
huh?
zzz
15 3 5 is what?
dr|z3d
1500, 300, 500.
dr|z3d
or just 1500, 750 (ssu1/2 combined).
zzz
ok
zzz
interesting: Java 20 is going to have optimized AVX-512 implemenations of ChaCha20 and Poly1305
zzz
zowie. testing throttler on ff
zzz
I dropped 1399 inbound conns in first 90 seconds
zzz
I think I'm onto something here :)
zzz
orignal, do you connect directly to ff to store your RI or do you send it through expl. tunnel?
dr|z3d
haha, very nice, zzz. "yes we can!" :)
zzz
ok -9 pushed, not perfect but better than before
dr|z3d
baby steps, as you're want to say :)
dr|z3d
so we're logging the rejections via the EventPumper?
zzz
test results: stats.i2p/docs/router.activePeers.png
zzz
pumper does ntcp, Est. Mgr does SSU
dr|z3d
rgr that
zzz
hard to miss
zzz
esp at startup
zzz
but you can see in the pic I dialed it right in at 15 minutes, at least for that ff
dr|z3d
yeah, see that.
zzz
non-ffs will look different and are more idiosyncratic
zzz
next on my list is to take a closer look at ff peer selector
dr|z3d
I think we need to drill down on what constitutes a good ff, and remove the ff flag from anything that doesn't meet the grade.
dr|z3d
U, transports disabled, K,L,M.. remove flag.
dr|z3d
and prevent force-floodfill from working if those conditions are met.
zzz
the "remove flag" is orignal terminology/concept. For us it would be simply tweaking the criteria for "bad"
dr|z3d
ok
dr|z3d
I'd be happier seeing the flag removed altogether so it doesn't show up in the console.
dr|z3d
but whatever works.
zzz
due to differences in the way we store RIs we can't do what you propose.
zzz
we have all the mechanisms and knobs we need in all the right places
dr|z3d
ok, no biggie. I thought because we can remove caps from router we could store those, but not worth worrying about.
zzz
just need to figure out which one to turn and how much
dr|z3d
so, flags aside, what do you think about hardening the requirements for ff?
zzz
for auto-FF? I don't see any reason
dr|z3d
the main motivations are twofold: firstly, more performant floodfills on the network by eliminating U and slow tier routers; secondly, increasing the barrier to entry by requiring both transports enabled.
zzz
I believe all that is in place now; please review current criteria first
dr|z3d
the requirements would cut both ways, both for auto/forced ff, and for how we grade them (ie bad or not)
dr|z3d
we don't require both transports enabled, re the rest, I think you're broadly correct, though we don't have any limiters on the forced-floodfill option right now.
zzz
any proposal must be w.r.t. current criteria and include an estimate of what % of current ffs would be eliminated
dr|z3d
ok, here's some stats: current ffs in netdb: ~1700, excluded ffs based on disabled transport (SSU): ~475.
zzz
and what % are java
dr|z3d
that I can't tell you :)
dr|z3d
what I can tell you is that most of those excluded ffs are probably hostile based on recent behavior.
zzz
your proposal would only affect java routers, so your estimate of eliminated ffs must take that into account
zzz
and I don't believe your facts are right re: our criteria, so please redo your research before making your proposal