zzz
I'm not interested in spending time optimizing the handling of bad IPs
dr|z3d
ok, that's also fine. just, you know, don't assume I'm out to undercut you or defenestrate proposals just because I ask a few questions :)
zzz
yeah I get it, it's fine, but when I'm trying to very politely make the case for something (as you wanted) I don't need a laundry list of ways I should really just work around it instead, and it's distracting to the person I'm asking, that's all
zzz
all good :)
dr|z3d
ok, sorry for the distraction. I'll try not throw boomerangs when you're on a conference call to orignal :)
orignal
will fix
orignal
but need to investigate it first
orignal
it's not supposed to be published
orignal
ofc it's a bug
zzz
good morning, thank you sir
dr|z3d
we've got an unknown unknown in the latest commit. another minor typo.
dr|z3d
/ TODO see if Alide RI will
dr|z3d
so minor it's probably not worth even mentioning, but still, see it, say it.
zzz
thx, got another checkin coming anyway
zzz
nice demo of bob asking multiple charlies, it goes pretty quick:
zzz
08:11:05.948 Charlie response 68 picked a new one [2605:a140:2085:2681:0:0:0:1]:20027
zzz
08:11:06.132 Charlie response 68 picked a new one 81.207.198.25:22225
zzz
08:11:06.269 Charlie response 68 picked a new one 65.21.108.14:12347
zzz
08:11:06.434 Charlie response 68 picked a new one 82.146.63.7:60111
dr|z3d
latest dupe hop request from: F1uJJB65ieZP-QjYgzmJQ5jcnCEG2tTBTFRW7QbBZJQ= (i2pd)
dr|z3d
that's pretty good, almost instant.
dr|z3d
(the bob/charlie flow)
zzz
orignal, FYI, I'm now bundling the RI with both relay intro and peer test msgs 2 and 4
zzz
and if it doesn't fit, I've added a delay so the RI should get there first
zzz
but it seems to fit about 90% of the time
orignal
I have added the code to wait anyway
orignal
for RelayIntro
orignal
back to ::1
orignal
it's more complicated
orignal
basically it happens when people enable i2pv6 in config but don't have actual ipv6
orignal
the right answer it to not publish ipv6 address at all
dr|z3d
that's it, orignal. no public ipv6, don't publish.
dr|z3d
there's a list of bogoon ipv6 masks here if that helps: freenetworktutorials.com/ipv4-and-ipv6-bogon-address-list
orignal
yes but how do I recognize it?
orignal
my assumption was that users are not complete idiots
orignal
but I was wrong
dr|z3d
wrong assumption. assume the worst and you can't go wrong :)
orignal
why 68 is so often?
zzz
I guess a popular peer in that example
orignal
but you tried 4 different
orignal
and all of them returned 68
zzz
I mean alice was popular, connected to all the charlies
orignal
how many links did she have?
zzz
don't know, I was bob
orignal
any idea how should we handle this "problem 68"?
orignal
because I see it all the time on my active floodfills
orignal
compares with number of existing connections?
orignal
I mean it's real problem now
zzz
I'm trying up to 8 charlies now. I haven't looked to see if that's ever not enough, but in the example above I only needed 5
orignal
I mean at Alice's side
zzz
well, peer test was never guaranteed to succeed every time
orignal
maybe we shouldn't run peer test if we see incoming sessions recently
zzz
I don't think it's a big problem
zzz
sure, you can do that
dr|z3d
aside: seeing 30 ::1's in a blocklist now I've added it explicitly. And a huge number of "not reachable on any transports".
dr|z3d
(165 unreachables)
dr|z3d
total in blocklist: 465.
orignal
I'm asking about possible strategy
orignal
say if we have more than 500 session and some incoming in last 2 minutes we don't run peer test
zzz
yeah we used to do something like that. We run test more often if we think there's a problem
orignal
I run every 41 minutes or something
zzz
we run every 13 minutes normally; 5 minutes if the last test was UNKNOWN; 45 seconds we are currently UNKNOWN or the last test returned something different and we want to rerun to confirm the change
zzz
and twice as fast if we're testing both v4 and v6
zzz
interesting dr|z3d. ::1 ended up on my sybil blocklist 4 days ago from the banning of all addresses of one router
zzz
perhaps we should add it; the downside is that you're blocking a router if only one of its addresses is bad
zzz
which we do for 127.0.0.1 now because it is on the blocklist
dr|z3d
yeah, I figured that it would block a router with potentially valid ips, but it's still a poorly configured router, so it's a toss up.
dr|z3d
hopefully orignal's on it so they become few and far between real soon now.
orignal
I will fix ::1 issue shortly
zzz
the 'ban all ips when you ban a router' is a little tricky because you can publish a bad ip in one address and the IP of the guy you hate in another, then get yourself banned and take out the other guy
dr|z3d
good boy, orignal *pat pat* :)
dr|z3d
hmm, interesting point, zzz.
dr|z3d
so can the banlist be tweaked to check for *any* valid ips before banning an invalid ip?
dr|z3d
I guess I mean the blocklist.
dr|z3d
the check for valid ips would presumably only occur for bogon addresses, otherwise public ips in the blocklist wouldn't undergo further checks.
zzz
I;ll test adding these to blocklist.txt:
zzz
+wildcard:0;0;0;0;0;0;0;0
zzz
+localhost:0;0;0;0;0;0;0;1
dr|z3d
the second one can be reduced to ;;1
dr|z3d
have it working like that. commented as "Bogon" here.
zzz
yeah guess it does, it's in my sybil blocklist as ::1, but the comments at the top don't say one way or the other
zzz
guess I'm kinda testing it anyway since sybil caught it on one router
dr|z3d
it expands to the full address as above when appearing in the console banlist
zzz
not sure why sybil didn't find all 150 routers on the same IP? maybe a bug somewhere...
dr|z3d
not floodfills? or you're talking about the results page?
orignal
fixed
dr|z3d
chocolate potato for orignal :)
orignal
ipv6 address will not be published until it receivs external addresses from SSU2 or specified explicitly
zzz
woot, rewrote the IP string-to-byte code and got a 10x speedup
zzz
like some freshman programming class
dr|z3d
great, zzz. next up you'll be announcing you got masks and ranges working for ipv6! :)
zzz
doesn't always work out, last time I had a great idea it ended up 10x slower
zzz
thanks again orignal
dr|z3d
makes it all the more rewarding when something does work :)
zzz
ipv4 481 ns -> 45 ns; ipv6 128 ns -> 13 ns. Not sure why ipv4 is slower even though string is longer, probably *= 10 vs << 4
zzz
(decimal vs. hex)
dr|z3d
nice speedup.
dr|z3d
yeah, and maybe computer likes hex better.
zzz
no its that shifting by 4 (to get *= 16) is much faster than multiplication by 10
zzz
barrel shifters are blazing fast
dr|z3d
I'll take your word for it :)
zzz
java is really fast if you don't use any of the java stuff
dr|z3d
haha
dr|z3d
the paradox.
dr|z3d
speaking of fast, I've been looking at FNDF and how it handles expired/invalid RIs.
dr|z3d
might be my code, made a bunch of changes there a while back, but removing the lookups for RIs we don't need to update brings job lag down noticeably.
dr|z3d
Job lag84 μs
dr|z3d
that's on one of SC's outproxies.
dr|z3d
not an outlier value, eiter. just dropped down to 70.
dr|z3d
message delay @ 4ms.
orignal
Routers: 8101
orignal
we need to reduce expiration time at floodfill
dr|z3d
router count's all over the place, orignal. some very noticeable ongoing spikes occurring daily.
orignal
yes
orignal
we just need to change our parameters
dr|z3d
I grade RIs based on quality. the better the quality, the more time they remain in the netdb.
dr|z3d
crap RIs are memory only, don't even bother writing those to disk.
orignal
guys, do we drop RI from future?
zzz
yes
dr|z3d
yeah, especially terminator routerinfos from 2100.
orignal
what's your threshold?
zzz
2 minutes
orignal
thanks
dr|z3d
you'll want to make sure it's fudge, orignal. every router needs fudge.
dr|z3d
if (routerInfo.getPublished() > now + 2*Router.CLOCK_FUDGE_FACTOR) {
zzz
ok, close-ipv6 detection is unimplemented in sybil analysis
dr|z3d
ah.
dr|z3d
I'm guessing ipv6 isn't your favorite thing :)
zzz
thats why they all don't have a thousand points
orignal
how?
orignal
you can obtain bunch of ipv6 address from different ranges easily
dr|z3d
he's talking about the sybil detection, orignal, and the non-implemented ipv6 element.
dr|z3d
maybe when you feel inclined you should fire up java i2p and see what he's up to :)
orignal
I'm talking the same
dr|z3d
ok
orignal
I can have two tunnels with two completely different ipv6
orignal
tell me how many routers do I run at home? ))
dr|z3d
3, or 4 at the weekends.
dr|z3d
:)
orignal
wrong ))
orignal
7
dr|z3d
:)
dr|z3d
so you got a java flava router running is what you're saying?
zzz
orignal, we have a "too close" detection for IPv4, like same /24 and same /16 ... we'd need something similar for v6
orignal
I know what I'm talking about
orignal
I'm just telling that nature of ipv6 is different
dr|z3d
I was asking if you'd seen what zzz's got going on with the sybil detection is all. Not disputing your knowledge :)
orignal
ipv4 more or less depends on physical location
orignal
ipv6 doesn't because tunnels over ipv4
dr|z3d
70 of 220 banned routers ::1 on one router.
dr|z3d
that's quite the hit rate.
orignal
NetDb: RouterInfo is from future for 85144 seconds
orignal
nice
dr|z3d
you could rephrase that to something like: NetDb: RouterInfo [truncated hash] published x seconds in the future, banning.. or thereabouts, orignal. reads better.
orignal
probably
orignal
but I don't care I just delete it
dr|z3d
sure, it's just logging. not super important.
orignal
just see few of them so far
dr|z3d
yeah, not frequent, but you might occasionally see a burst of them.
orignal
I suspect my netdb is full of them for a long time
dr|z3d
we do the same with RIs published in the past.
orignal
need to check leasesets
dr|z3d
let's see..
orignal
since tgey also contain timestamps
dr|z3d
return "Router [" + routerId + "] is SSU only without introducers and was published over 45 minutes ago";
dr|z3d
return "Router [" + routerId + "] is unreachable on any transport and was published over 45 minutes ago";
dr|z3d
I've got a few more cases where routerinfo is deleted.
dr|z3d
return "RouterInfo [" + routerId + "] is K or L tier and was published over 4 hours ago";
dr|z3d
String minRouterVersion = "0.9.20";
dr|z3d
return "Router [" + routerId + "] is too old (" + v + ") - banned until restart";