zzz
0) Hi
zzz
hi
eyedeekay
hi
zzz
what's on the agenda for today?
zzz
you have a go-i2p status report?
eyedeekay
I don't really have anything ready for go-i2p report yet
orignal
hi
orignal
network overload
zzz
ok network overload will be 1)
zzz
I'll add:
zzz
2) congestion caps
zzz
3) new proposals
zzz
4) two leasesets from postman
zzz
anything else?
zzz
ok then
zzz
1) network overload
zzz
go ahead orignal
orignal
it's just overloaded again today
orignal
Ilita has reached 100 Mbs
orignal
does someone have some new info where it comes from?
zzz
no
orignal
also people complain that tunnel build ratio is low on active node with a lot of transit
zzz
but I promised I'd give bitcoin an update "mid Jan." so I need to do that
orignal
for me it looks an attack
orignal
inteninal
orignal
to put the netwrok down
zzz
by my stats we've been in congestion since early sunday
orignal
R4SAS reported his router has reached 15K ransit tunnels limit
zzz
things were improving all last week
orignal
we see it bad today
zzz
I did some research yesterday
zzz
to see how various versions were handling the traffic
zzz
so I did a plot of part. tunnels by router version
orignal
yes last week
zzz
java routers only ofc
orignal
but now it's worse again
zzz
the 0.9.57 routers are handling the storms very well
orignal
and what's your conclsion?
zzz
0.9.57 has 25% less part. tunnels than 0.9.56
zzz
because it's properly rejecting
zzz
also, all routers <= 0.9.56 see the storms a lot worse
zzz
here's my new chart
zzz
so my conclusion is that my network stats are actually quite promising... 0.9.57 routers are not having trouble
zzz
it's all the older routers that are dealing with it and affecting the stats
zzz
what do you think?
orignal
then why tunnel creating rate is low?
zzz
because everybody is filling up with spam tunnels
orignal
my position is the same as before, we can handle it
orignal
but we don't like that tunnel build requests get rejected too often
orignal
I have change my profiling
orignal
I give up a router for 2.5 minutes after reject
zzz
we're still only at 44% 0.9.57
zzz
yes, it's a good time to work on profiling and peer selection code
zzz
I have some changes coming also
orignal
and also for rouyers is no reponse too often
zzz
right
orignal
plese tell me if you ban a router if you receive tunnel build requests too often
zzz
we "blame" peers when no response. it could be that tunnel or the reply tunnel, we add them up and say, 33% blame to him, 15% to her, 50% to that guy, etc
zzz
so that all goes into the profile
zzz
we do not ban a router for that, I think it's a vulnerability to do so. drz was doing that and I talked him out of it
dr|z3d
orignal: I do, but I have a much more generous allowance for requests than zzz.
dr|z3d
I barely see any bans these days.
orignal
I mean not router but tunnel build coming from it
orignal
see what happend
zzz
yes we only any router to be in 3% max of participating tunnels (more or less)
orignal
low active router expecially firewalled has much better rate
orignal
than high loaded router
orignal
it simply means you punish routers for active particiaption
dr|z3d
I think my rate is now at something like 15 (?) requests every 220s for slow routers, more for faster routers.
dr|z3d
with a sliding scale based on overall percentage.
orignal
zzz, not a problem with transit tunnels
zzz
ok good
orignal
the problem is own tunnels
orignal
that an active router can't build tunnels
dr|z3d
Haven't seen anything explictly banned in a while. Very rarely do requests get dropped.
zzz
sure. in times of congestion, slow down tunnel build, don't speed up
zzz
go down to 1 tunnel
orignal
maybe you guys drop requests too often
zzz
we're handling 50% more tunnels per router than back in November before this all started
zzz
we can handle it
orignal
as result I will have to change my profiler to bypass routers that don't let me build my tunnels
dr|z3d
any theory on the asymmetric traffic, zzz? it's sometimes pretty pronounced.
orignal
and if Java keeps droping but i2pd doesn't
zzz
I still think it's tunnel builds from insane sam clients or i2pd routers, but it could also be i2p plus infinite loops or bugs
orignal
it will endup that i2pd build tunnels through other i2pd only
orignal
also I see not extra transit on ygg-only routers
zzz
yeah we don't want to segment the network into two parts
zzz
it's just going to take some time to work out
zzz
anyway, what should I tell bitcoin?
orignal
and since Java never selects ygg-only routers for tunnels
orignal
you should tell there is nothing to do with them
dr|z3d
orignal: that's always a good idea.
dr|z3d
if you get repeated rejects from a router, assume it's either overloaded or doesn't like you and avoid it for a while, don't keep sending requests.
orignal
also I saw sormycloud's tweet that he has bandwidth issue
zzz
yeah I think I have to say that, sorry if the GH issue sounded panicky, we're smarter now, probably not bitcoin
orignal
dr|z3d but it will endup with split queicly
orignal
zzz, ofc no
zzz
stormy is running i2p plus and I think is vulnerable to several nasty bugs
orignal
the way how they implemented is shitty
orignal
and they should fix this issue regardless
zzz
right orignal we want to encourage them to fix it, they are doing that
zzz
agreed
orignal
aboyt strormy
orignal
if he has bandwidth issue
orignal
maybe too many people use i2p as tor
zzz
ok that will be my message, I'll put it in the GH issue later this week
zzz
anything else on 1) ?
dr|z3d
not that, orignal
zzz
go ahead about stormy
dr|z3d
outproxy traffic is what you'd consider normal.
dr|z3d
if anything, the issue is probably Tor itself and the ongoing DDOS.
orignal
however
orignal
is it possible that bunch of people use i2p instead tor?
dr|z3d
we did see some huge usage on one of the outproxy routers before Xmas.
orignal
expecially from Russia
dr|z3d
but that wasn't outproxy traffic.
orignal
so would be nice what stormy says
zzz
I've also added outproxy warnings to our bittorrent and embedding guides
dr|z3d
I'm just talking to him. He just wants faster is all. Doesn't sound like an issue as such. grumbling about needing a 10Gb pipe.
zzz
it's not directly related to request rejects
orignal
fine for me
eyedeekay
I don't have any other concerns you haven't been able to address
zzz
other than to avoid getting there
zzz
a couple issues with implementation:
zzz
a) I can't enable it now or we find all the dev build routers
dr|z3d
and if I'm publishing a "high loaded, avoid" cap and you're still pushing requests my way, then you're probably up to no good if you known about the caps, no?
zzz
b) I don't really want to do the profile-update side of it until a lot of routers support it
zzz
so I think what I want to do is: Flip a switch to turn it on after our next release is out
dr|z3d
re a) just do what you've done for the simple stats, zzz.
zzz
and then do the profile-handling part of it in the release after that (July)
orignal
fine
zzz
I'll put one router up now that will publish it, since it's in a dev router family anyway
dr|z3d
if local.version < 0.9.58, don't enable. that's the basic logic, orignal
zzz
we good with D/E/G or anybody want to spend a week arguing about it? :)
orignal
so should I add code just to recognize them?
orignal
why? If see such code it means I can use it
dr|z3d
D E G seems fine, no sense bikeshedding.
zzz
you can if you want orignal, I'm worried about adding a bug that would avoid everybody and break things
zzz
so I may recognize it but not do too much with it
orignal
I mean just to not produce log message if I see such unknown code
zzz
that's the hardest part, if everybody publishes E, don't break
zzz
sure
zzz
thats fine
zzz
I'll put up my router with it tomorrow
zzz
and I'll put this into a proposal, which will be 162
zzz
anything else on 2) ?
orignal
ok. 162
orignal
no
zzz
3) new proposals
zzz
so I have three stubbed out and loaded up
zzz
162) congestion caps
zzz
163) Datagram2, to fix the signature issues from back in the LS2 proposal
zzz
164) Just a stub, it's the issue I'm working on privately with orignal, will make it public later
orignal
good point
zzz
and I'm awaiting to test that fix with you when you get to it, no rush
orignal
yes I will implement 164 soon
orignal
also what do you guys think about quic instead streaming?
zzz
so I'm putting them all on the website, but 164 will only say 'streaming', thats it, nothing else
zzz
SSU2 nearly killed me, let's not do it again :)
orignal
I should
orignal
because streaming becomes a bottleneck now
zzz
anything else on 3) ?
zzz
maybe it just needs some tune-ups/tweaks/fixes
orignal
I plan to implement UDP socks proxy
zzz
client-side you mean?
orignal
no, streaming alsways use one tunnel rather than all available
orignal
yes, client proxy
zzz
oh, that's the multipath TCP stuff, that's really hard, I looked at it once
orignal
you meantione you have it already
zzz
right
zzz
anyway
zzz
anything else on 3) ?
orignal
no
zzz
4) two LS from postman
zzz
I spent about 15 minutes looking at it on stats.i2p which has a similar setup and looking at logs
zzz
didn't see anything wrong
zzz
all looked fine and only got one LS from stats
zzz
I'm awaiting an answer from postman on whether he's on the i2p plus platform or not
orignal
I can tell you more
zzz
once I get the answer, I'll either forward that info to dr|z3d or dig into it more myself
orignal
postman always sends back wrong one
orignal
and I have to re-request it from floodfill
zzz
that makes sense
orignal
hence get disconnects from Irc2P too often
zzz
yeah that doesn't help
zzz
not a bad problem, but not great either
dr|z3d
related, being able to expire leasesets locally before they're expired, to force an update.. worth thinking about?
zzz
so when I hear from postman we'll move forward
zzz
dunno what you're trying to accomplish with that
zzz
anything else on 4) ?
dr|z3d
a nicer way of requesting a new lease than restarting the client tunnel is what I'd hope to accomplish.
zzz
well you could put a delete button on the console I guess
orignal
that's what we do now
dr|z3d
yeah, that's kinda what I'm driving at, zzz.
orignal
no
zzz
have fun
zzz
ok, next meeting in one week or two?
dr|z3d
delete remote lease, or delete all remote leases. :)
orignal
2
zzz
ok, Feb. 6
zzz
thanks everybody
dr|z3d
fwiw, zzz, while I'm still trying to work out the stackoverflow bug, for now I've backed out a chunk of the ISJ mods.
zzz
nice
dr|z3d
so we'll see if that helps anything.
zzz
I'm thinking of a more structured way to try to convince you of where you've gotten in the weeds, without the pain or time of a code review
zzz
maybe next week
dr|z3d
I'm still varying concurrency and timeouts based on a bunch of criteria, but otherwise, everything else is as canon.
dr|z3d
when I've confirmed it's not breaking stuff I'll commit it and invite you to check it's more sane :)
zzz
you're not grasping how deep down in the router you are in ISJ or what the rules are when you're there
zzz
and you're probably doing it elsewhere
zzz
OOMs, heisenbugs hard to reproduce, network spam, or problems in unrelated areas could be the result
dr|z3d
sure, can't argue.
dr|z3d
ok, this patch seems fine on a few routers, no obvious epic fail happening. let me commit it.
zzz
well, your mods are not (at all) functioning as intended, so they weren't well tested, and rest assured there's no downside to ripping them back out :)
weko
Tunnel creation success rate: 22%
weko
Looks bad
dr|z3d
I have mixed results on various routers. one is 55%
weko
Client Tunnels: 295 (I haved 220 some days ago)
zzz
that change is insufficient, as you've left in near-identical borked code in the constructor that was implicated in the stack overflow last week, but now I'm just repeating myself and I can't be your code review guy
weko
My router always have >9000 tunnels
weko
zzz: are you really working on multipath streaming? I guess оrignal suggest you, I am right?
dr|z3d
boolean uninteresting the problematic section, zzz?
dr|z3d
oversight if it is, I meant to remove that.
zzz
so no, it's not all as in canon
zzz
your constructor mods were directly in the path of the stack overflow last week, check the logs, and if it happened once I bet it's happening to lots of people
zzz
I don't know how you thought you fixed it last week but your code is still in there
zzz
identify the loop, find out which of the steps is your mod, rip it out, that breaks the loop, that was my advice last week
dr|z3d
if that doesn't fix it, I'm at a total loss as to what does.
zzz
but my advice this week is rip it all out
dr|z3d
aside from modifying concurrency and search timeouts, I have.
zzz
I'm not even gonna look, that's all I got for now
dr|z3d
if there's something obvious in the latest commit that's still not been addressed, I'd be happy to know what it is.
dr|z3d
great. thanks. remind me to be equally elliptic when you need help :)