~dr|z3d
@RN
@RN_
@StormyCloud
@T3s|4
@T3s|4_
@eyedeekay
@orignal
@postman
@zzz
%Liorar
+FreefallHeavens_
+Leopold
+Xeha
+acetone
+bak83_
+cancername
+cumlord
+hk
+poriori
+profetikla
+r00tobo
+uop23ip
+weko
An0nm0n
Arch
Danny
DeltaOreo
DrSekra2
Irc2PGuest53061
Irc2PGuest53871
Irc2PGuest89421
Meow
Nausicaa
Onn4l7h
Onn4|7h
Over1
T3s|4__
anon1
anu3
boonst
khb
mareki2pb
not_bob_afk
shiver_
simprelay
solidx66
thetia
u5657
zzz
eyedeekay, re: ls spam, have you found any existing limits or mitigations? I haven't
zzz
dr|z3d is a sidebra for your sideboob? ))
eyedeekay
No nothing that works as a rule every time without doing something active/resource intensive to back it up(like try to fetch the leaseSet in another client and see if something is there or if it's just fakery) none of which seems reasonable
zzz
I also preliminarily conclude that it would be tough for us to implement some of orignal's ideas given our architecture that separates streaming over on the client side
zzz
and the checking-before-store would be inefficient and has the side effect of dropping new LSes not old ones when under pressure
zzz
I have a gut feel we'd struggle with 4 MBps of leasesets, doesn't seem realistic, but not sure if they would get throttled somewhere in general-purpose queue management / RED code
zzz
thoughts?
eyedeekay
By check before store do you mean try to ping the new LS or orignal's idea of comparing communications that have happened before to potential new ones? Either way I think it's probably unworkable
eyedeekay
I had thought that maybe we could do some kind of cross-comparison, like if a LS appears in more than one DB use the information from the older one(presumably contacted?) to verify the newer one
eyedeekay
But that would only work if a LS appeared in at least 2 subDbs
eyedeekay
I tenatively disagree with orignal on capping the number of LS's per DB but I see his point, perhaps instead of capping the number limit the growth
eyedeekay
In client DB's anyway, not sure about the router-wide DB
eyedeekay
But even on a fairly active snark client I still see less than 200 or so at a time, I think limiting the growth/second is probably not harmful
eyedeekay
It would be simple if we could just conclude (X is sending too many LS's, this is unreasonable don't accept any more from X until they stop) which is maybe happening in queue management or RED code as you say but not in a specific and conclusive way that says "LS spam is going on, let's deal with it"
eyedeekay
Maybe that's better than specifically responding to spam though, if the system that is handling it in a general way can handle this specific thing
zzz
check-before-store means use some criteria before storing, maybe only if under pressure
zzz
agreed doing some additional fetches to try to 'verify' is unworkable
eyedeekay
Yeah it seemed to me that would just amplify the attack during the attack phase with everybody reaching out to verify the fake LS's
zzz
so I found some WIP code to do 'aggressive expire' in ExpireLeasesJob, similar to what's now in ExpireRoutersJob and working very well
zzz
so that's a post-store removal if 'too many', and would probably remove oldest-first
zzz
orignal reported 220K LS so that's 22K/minute or about 2MB, not enough to OOM
zzz
that would be efficient because it would only kick in if over a limit, not something we do on every store
eyedeekay
Hm, that makes me wonder re: ExpireLeasesJob, one of the things to think about would be how we select which ones to expire
zzz
it's a fairly blunt and dumb mitigation for a fairly blunt and dumb attack
eyedeekay
But I agree being more aggressive about expiring them after receiving them does seem sensible
eyedeekay
And after we've had them for a while maybe we have enough information to at least know we aren't tossing good ones
zzz
oldest-first for client dbs and non-ff, out-of-keyspace for ff db
zzz
what information are you thinking?
eyedeekay
Well if the spam LS's don't correspond to real clients then we will have never managed to contact them or received anything from them
eyedeekay
So if we can keep track of who we've actually talked to then we know not to throw them out
zzz
we don't track anything like that anywhere afaik and don't know where we would
zzz
any mitigation that requires us to 'track more things' (growth/second, contact time, usage time, ...) is by definition expensive/complex
zzz
eyedeekay, just assigned you another ticket, and this is your monthly reminder to please address your tickets, don't wait to the last minute
eyedeekay
Responded, I think I may have already checked in the fix and asked OP to try a dev build
eyedeekay
re: track more things yeah and we keep lots of DBE's around which is where "hasBeenContacted" or whatever would have to go so even if it's a boolean it's like ~2000 booleans
zzz
ok, well I've never had much luck convincing you of this, but let's follow real processes. If you fix an issue, put the issue number in the checking comments, and close the ticket
zzz
but last time I looked you still had HTML errors, so please take another look
zzz
and not sure it looks that great either. you made quite a mess of the render leaseset code, do what you can to get it right
eyedeekay
Oh jeez I think I've got the HTML stuff sitting in MR 219
zzz
you should also look at the actual line from the NPE vs. the 2.7.0 code and verify you fixed it, not just ask the guy to retest
zzz
let's be serious about fixing regressions
zzz
I haven't seen MR 219, nor have you added me as a reviewer, and remember I don't get notifications
eyedeekay
Ack, the HTML gets fixed by removing the extraneous closing tags in MR 219
eyedeekay
I checked the line corresponding to the NPE in 2.7.0 and it is the fix I checked in before
zzz
ok good. I just approved 219, taking your word for it
eyedeekay
Thanks, merging it now
zzz
I'll resurrect my aggressive expire WIP. It will be annoying to test but I'll figure it out
zzz
look for an MR next week
orignal
my idea is to check LS's static key with session's if it's unsolicited
orignal
e.g. peer can only send thier own LS
orignal
unless it's requested explicitly
orignal
and no I never had as idea to capping out number of LSes
dr|z3d
haha zzz, yeah, the shame :)
dr|z3d
*** blames orignal ***
orignal
what?
dr|z3d
just rattling your cage, orignal :)
orignal
so what we LS spam?