#saltr (irc2p) | I2P+ 2.5.2+ >> i2pplus.github.io | skank.i2p | git.skank.i2p | purokishi.i2p | vituperative.github.io/i2pchat | natter.i2p | notbob.i2p | ramble.i2p | shreddit.i2p | cake.i2p | /msg an op for voice

obscuratus orignal: I think that's what we're intending to do already. If the Inbound Message Distributor receives a DSM with forwarding instructions (that's delivery type Router, right?) coming down a tunnel, we going to drop that message.

obscuratus Or, at least, I've proposed that. It hasn't been merged, but I think it's the same thing as you're talking about.

obscuratus orignal: I had thought I2PD already did that, no?

orignal ofc we do for IB

orignal but I'm talking aboue OBEP

orignal if somebidy send it through an aoutbound tunnel

obscuratus Heh! I guess I'm still learning. I thought the OBEP would only talk to IBGW. I didn't realize it would do that. :)

orignal how do you publish you RI to an FF if you can't reach it directly?

orignal the only way is through tunnel

dr|z3d iirc, java does non-encrypted datastores for slower (arm/android) routers, but I may be wrong.

dr|z3d ofc, we could ensure they encrypt.

obscuratus orignal: Yeah, I think we send it out an exploratory tunnel.

obscuratus So, this proposed change would then eliminate the ability to send your RI down an exploratory tunnel, right?

obscuratus Or, does it arrive at the OBEP as a garlic message?

obscuratus orignal: You've got me curious. I'll add some logging.

orignal gents you ban real routers

eyedeekay Yeah it's obviously happening, we need to get the fake ones out of the netDB before the sybil analysis sees them, working on it

eyedeekay We can at least cut this down significantly I think

orignal as result tunnel creation rate is low when even no attack

eyedeekay Yeah it's resulting in observably fewer floodfills even after an attack because we have to wait for bans to expire, we could think about expiring them faster but I'd rather try to stop the false positives

eyedeekay Oh great, looked at my graph, just saw it climb from 1200 to 1800 then the bans started

eyedeekay Let's see where this goes over the next 20-30 minutes

orignal no, ther are attacking duiring our nighttime only

eyedeekay Hm. Wonder why I would be seeing it change like that then

eyedeekay Never mind, stabilized around 1650 floodfills known to me, real or not

orignal they found out that you, me and dr|z3d are in EST

mesh orignal: yeah they attack during a specific time winow

mesh I sort of wonder why

mesh what's in the latest i2p+ update that went out today? (why is there no changelog... blog entry... argh)

orignal two theories

orignal then want to attack when we are offline

orignal or it just thier working hours

mesh (when you're offline)

orignal during my night of course

eyedeekay Those are not mutually exclusive

mesh but yeah. the attack can be sorta mitigated simply by restarting the router

mesh I've also found the attack can be mitigated by creating a very agggressive flood fill

orignal there keep changing strategy every day

orignal *they

mesh orignal: for the past two days they've gone with the forged ri attack

orignal eyedeekay what do you think about forbidding database store message on OBEP?

orignal they forged them differently

mesh it's quite effective and is likely quite cost effective because it tricks good routers into doing all the work

orignal initially they just point whole rouyter content

orignal now they chnage s key

RN the sybil test is part of mesh's strategy, eyedeekay what is going wrong when the sybil test gets false routers?

orignal this night they clone only particular floodfills

mesh orignal: why does that matter...

obscuratus RN: I tricks Sybil into banning routers and whole IP addresses.

mesh the attack can still be mitigated by creating a very aggressive floodfill router

obscuratus How do you make your floodfill aggressive? Testosterone shots?

mesh I upped the exploratory tunnel count to 32 (even though the webconsole maxes out at 16 tunnels), I decreased peerTestTimeout to 250, I run sybil every hour but actually raised the minimum threshold for block to 50, and of course force all the routers into being floodfills

eyedeekay Yeah that's not a thing I'm familiar with either, can you refine your terminology? Like are you saying "manually enable floodfill" or "offer lots of bandwidth" or "use restrictive sybil-analysis settings" or something like that?

eyedeekay Oh never mind you explained

eyedeekay Thank you

mesh the result is a router that is very well integrated into the network. The share ratio is cut in half but it resists the forged RI attack. During today's attack my bsr never dropped below 40%. All my i2p web services continued to be available

orignal asnother thing we have noticed

orignal they copy family signature with all field

mesh it sort of makes sense: if your router is very aggressive in exploring the network and build up its own authoritative view of the network, it's resistant to being flooded with fake news about the network

orignal and of course it fails for fake routers

orignal just FYI

eyedeekay IMO a failed family sig should be an immediate ban, no reason for it that is non-malicious

orignal yes, what I did now

mesh eyedeekay: what about the stormycloud routers?

orignal so for strimycloud your alsways know which one is real

eyedeekay Fair enough, there is a model defense laid out on the netDB page where everybody manually enables floodfill, as "fight sybil with more sybil" yours is an interesting take on that

orignal one with good signature

eyedeekay Bad sig is a bad sig, if it's a mistake on StormyCloud's part, they should fix it but in all likelihood it's an attack

orignal they have fixed long tim eago

orignal if I remeber

eyedeekay Good

orignal but clones of strormycloud routers contain original one

orignal that's obvously fails for another router ident

mesh eyedeekay: yeah exactly. aggressive floodfills are very difficult to be knocked offline. During the attack I saw virtually no degradation in the availability of my I2P services. In fact they seem to be better performing because I reduced peerTestTimeout to 250. The tradeoff is that the routers with this configuration (1) consume a lot more cpu resources and (2) they share a lot less bandwidth because most of

mesh the bandwidth is consumed building exploratory tunnels

orignal ofc in few days they will discover it

mesh eyedeekay: you know the attack happens so suddenly that I wonder if the router couldn't detect it. Today I saw floodfill count jump by thousands in less than an hour

mesh one simple heuristic would be to track the number of floodfills that are all showing up wwithin a 60 minute window and punish them

mesh what I always notice (I'm usually awake during the attacks which happen in my mornings) is very quick rise in floodfill count followed by a very quick rise in the banned count. Banned count can climb very high -- 20,000+. Usually it sits below 2,000. From a heuristic perspective it's very clear that things are not normal and the network is under attack

mesh orignal: btw are RIs signed by the floodfill that publishes them?

orignal no

orignal only by themselves

mesh orignal: so there's no way to tell the provenance of an RI? like which floodfill it came from?

orignal no

mesh hmm yeah that's a problem

mesh even if we can heuristically identify the suspicious floodfills we can't ignore their forged RIs can we

obscuratus orignal: Regarding dropping RI DSM at OBEP, we would want to check and make sure we actually have code that does this, but didn't realize it.

obscuratus If this is something we never do (canon and cannon), and something you never do, then when we see it, it seems like it could only be malicious.

orignal no you shouldn't

dr|z3d in java, we can validate a router's ip address with geoip.

mesh dr|z3d: it's not exactly 100% accurate

obscuratus orignal: That's why I modified my logging. I wanted to check on my testing network, and see if I ever get those.

mesh dr|z3d: but it could definitely become another factor

obscuratus How would we use GeoIP?

dr|z3d at this stage, best effort trumps 100% accuracy.

dr|z3d we check if the published ip address for the router validates via geoip, obscuratus. if it doesn't, ban the router.

obscuratus Oh, so you just checking if the IP is somewhere in the range of the GeoIP list?

dr|z3d that ends up pre-empting sybil bans, so we retain most of the valid floodfills.

mesh dr|z3d: you might restrict such a check to floodfills. it would be another constraint on floodfills.

dr|z3d you query geoip directly, obscuratus. getCountry() or thereabouts.

mesh dr|z3d: I don't rememmber if the fake floodfills had valid geoips but it seems reasonable

orignal but now it's useless becuase fake routers has real IPs

mesh dr|z3d: also as I mentioned above, all these fake floofills come online at the same time. I would recommend punishing floodfills that are themselves part of a flood. perhaps more than X other floodfills all joined within 30-60 minutes

dr|z3d not useless, orignal. quite effective.

RN (side track: if major is a logging bot, why does it have voice?)

mesh (side track: if major is a logging bot we should burn it at the stake)

RN lol, be nice mesh.

dr|z3d we're looking up the transport ip, orignal, not the RI published ip.

dr|z3d and therein lies the weakness of the current attack.

dr|z3d see FloodfillPeerSelector, mesh, for an insight into how floodfills are grade.

dr|z3d *graded

dr|z3d obscuratus: public String getCountry(Hash peer) in CommSystemFacadeImpl

dr|z3d on a floodfill here, just shy of 1200 active floodfills, 7.5K banned routers.

dr|z3d build success at 70%

mesh I've got similar numbers. 1200 floofills with 8.6k banned routers

dr|z3d and sybils meeting the ban threshold? 0.

mesh dr|z3d: is there a place in the code where a new floodfill is detected for the first time?

dr|z3d as soon as a new RI is acquired.

dr|z3d as for blogging, mesh. no. sorry. don't have time for that. read the commit logs if you want insight into what's changed. don't be lazy, or entitled.

dr|z3d RN: more a status thing with major, but the occasional random intervention is amusing :)

dr|z3d as for targeting floodfills, current attacks aren't solely utilizing floodfills.

mesh yeah I think part of the problem is that even if you know all the bad floodfills you can't identify RIs

mesh that came from those bad floodfills

dr|z3d once you've banned the floodfill, you won't receive bogus RIs from said floodfill.

obscuratus dr|z3d: Until we address issue 397, we can't be sure that floodfill isn't forwarding that RI for someone else.

dr|z3d just set your global logging to warn, mesh, and enjoy the show.

dr|z3d that's a separate issue, obscuratus. all I'm saying is that banned floodfills can't forward RIs for anyone.

obscuratus They may be a perfectly legitimate FF, but we just forwarding the RI for someone else.

dr|z3d sure, good floodfills will happily send shit RIs all over.

obscuratus Yeah, until we fix 397

dr|z3d so really we need to address the issue at both ends.

dr|z3d we check RIs on receipt, and floodfills check RIs before send.

dr|z3d there's some laxity wrt to what floodfills will send, RI-wise. I've tightened that a little.

obscuratus I'm leaning towards orignal's solution. Have something like a white-list for known-real FF. The others will wait their turn until we can contact them.

dr|z3d in canon, routers that are considered perm-banned will be distributed by floodfills.

dr|z3d *aren't considered

dr|z3d sure, a whitelist could work, but we already have something along those lines already with the ff selector.

mesh what is 397? should floodfills republish and re-sign RIs? Seems like it would be good to know the true floodfill 'origin' of an RI

obscuratus Yeah, but so what. As long as the receiving router has a list of known-real FF, it will also wait until it has contacts that FF (or not).,

dr|z3d we already profile floodfills, so using that as the basis for whitelisting is what I'm driving at.

dr|z3d I'd be just as happy seeing a session-persistent blacklist for bad floodfills in similar vein to the sybil detection.

obscuratus If querying the peer profiles is fast enough, that might be a solution. But we might need a quicker in-memory method purpose built.

dr|z3d profiles for active peers are help in memory.

dr|z3d *held

obscuratus I'm coming around to something similar to orignal's view on banning. It really looks to me like some one or several people are actively expoiting flaws in our banning heuristics.

dr|z3d they're trying.

dr|z3d not getting very afar here, for reasons I mentioned above.

dr|z3d *far

obscuratus If they're causing good floodfills to be banned, they're succeeding.

mesh obscuratus: that is the nature of the attack

dr|z3d orignal's estimate is around 1.2K good floodfills, which is consistent with what I'm seeing on various routers.

mesh the attackers generate a lot of fake floodfills. they publish garbage. this is followed by dramatic rises in the banned count

mesh it's a cheap and effective attack I bet, more so than the previous attempts to ddos the network by sending lots of data

dr|z3d as I mentioned, obscuratus, sybil scans aren't banning anything here.

obscuratus I'm scrolling back, but can't find it. If I said sybil was the only banning problem, I mis-spoke.

mesh I see lots of "Floodfill with SSU disabled" "LU and older than current version"

obscuratus As it is, running once a day, I'm not at all sure it's effective for it's desired purpose.

mesh in the ban list

mesh very few say "Sybil Analysis" ... I assume those routers were banned by the sybil

obscuratus Do we even have a reason listed for our IP bans?

obscuratus The IP bans are probably our achilles heel with respect to banning good FFs.

dr|z3d blocklist, no, session bans, yes.

dr|z3d 1d? I can't remember if I tweaked I2P+ defaults, but locally it's @ 3h.

mesh mayhaps there should be an "unban all" button btw

mesh you can unban inividual peers but of course that's silly when you've got 20,000+ banned routers and many of them are actually good routers

dr|z3d if you've got 20K bans MOST of them are shit routers.

obscuratus I look at it from the perspective of controller theory. When you're trying to control a parameter with discrete measurements, the polling period needs to be much faster than the response time. Ideally, instantaneous polling.

mesh dr|z3d: I think there's lots of good routers mixed in with the bad routers

mesh for example, the "Floodfill with SSU disabled" probably are bad routers

obscuratus Which reminds me, we also need to address the issue we ran across a few days ago, where someone was spoofing RI that which we couldn't discern from real RI, and we're banning that router.

obscuratus So, as it stands now, easy, just spoof a FF router ID, but without the SSU transport, and we ban that router without realizing there's a good RI out there for that router.

dr|z3d I can only suggest you take a look at how I2P+ handles the problem, obscuratus, and then take a view on its efficacy.

obscuratus But, I feel confident we will fix that one.

dr|z3d good floodfills without SSU are few and far between.

obscuratus What if it really has a published SSU transport, but we're looking that the spoofed RI that wrongly says it doesn't have a SSU transport?

mesh I wish there was a way to record the attack and replay it. Now I'm genuinely curious to know which comes first.

RN LOL major

dr|z3d check for a valid ip before all else.

mesh dr|z3d: how to check for a valid IP?

dr|z3d only when the ip is proven valid (geoip-resolvable) do you then eliminate ssu-disabled floodfills.

obscuratus The spoofed RI's had the same IP as the real RI.

dr|z3d transport ip, NOT RI ip.

dr|z3d I pointed you at the code in question, obscuratus.

obscuratus The RI is how we know their transports.

dr|z3d so I shouldn't need to be explaining this :)

dr|z3d not the published ip, the ip being actively used I think is how we're doing the lookup.

dr|z3d /**

dr|z3d * Uses the transport IP first because that lookup is fast,

dr|z3d * then the IP from the netDb.

mesh obscuratus: this is where the whitelist would kick in?

obscuratus mesh: We're really talking about two separate problems actually. We'll probably fix the issue with spoofed IPs.

obscuratus The second issue may be able to be quickly resolved by tweaking our method of building a list of FFs.

obscuratus If the check of 'last heard from', or something like that, in peer profiles is quick, just add that screener to our ff selector.

mesh obscuratus: It sounds to me like the fundamental problem is: "I'm looking at a RI for a suspicious Floodfill. But before I ban this Floodfill how can I verify that the RI I'm looking at is not a forgery?"

mesh at least when you put it like that then the whitelist makes sense. that's just a reputation attack. but there are many floodfills who we can be pretty sure are not bad actors even if you see a suspicious RI that somebody published about them

dr|z3d it's fast, but we don't profile floodfills on sight, we profile them on a schedule.

dr|z3d or rather, we grade floodfills based on their profile data on a repeat schedule.

dr|z3d if a floodfill is too fresh, poorly responding, we put them at the bottom of the grade table.

mesh there are floodfills out there that are my homies. there's one estonia that stuck with me through thick and thin. I would never want to ban him just because I see a suspcious RI with his IP