~dr|z3d
                        
                        
                            @RN
                        
                        
                            @RN_
                        
                        
                            @StormyCloud
                        
                        
                            @T3s|4
                        
                        
                            @T3s|4_
                        
                        
                            @not_bob_afk
                        
                        
                            @orignal
                        
                        
                            @postman
                        
                        
                            @zzz
                        
                        
                            %Liorar
                        
                        
                            %acetone
                        
                        
                            +FreefallHeavens
                        
                        
                            +Onn4l7h
                        
                        
                            +Over
                        
                        
                            +Sh0ck
                        
                        
                            +bak83_
                        
                        
                            +bpb
                        
                        
                            +leopold_
                        
                        
                            +r00tobo_BNC
                        
                        
                            +uop23ip
                        
                        
                            +xHarr
                        
                        
                            Arch
                        
                        
                            BubbRubb
                        
                        
                            Danny
                        
                        
                            DeltaOreo
                        
                        
                            H20
                        
                        
                            Irc2PGuest16752
                        
                        
                            Irc2PGuest33667
                        
                        
                            Irc2PGuest97218
                        
                        
                            Maylay
                        
                        
                            Meow
                        
                        
                            ac9f_
                        
                        
                            anontor
                        
                        
                            b4dab00m
                        
                        
                            duck
                        
                        
                            gellegery
                        
                        
                            halloy13412
                        
                        
                            john231
                        
                        
                            makoto
                        
                        
                            nZDoYBkF_
                        
                        
                            nilbog-
                        
                        
                            ntty`
                        
                        
                            poriori_
                        
                        
                            profetik1
                        
                        
                            r00tobo[2]
                        
                        
                            shiver_
                        
                        
                            simprelay
                        
                        
                            solidx66
                        
                        
                            thetia
                        
                        
                            u5657
                        
                        
                            vivid_reader56
                        
                        
                            zer0bitz
                        
                    
                    
                        zzz
                    
                    
                        looks like from here, the attack either was modified or stopped about 12 hours ago; plesae report observations
                    
                
                
                    
                        zzz
                    
                    
                        I have zero bans remaining of suspect fakes
                    
                
                
                    
                        zzz
                    
                    
                        there's an ongoing background noise of week-old RI store attempts, going to try to track those down and figure out why they are rattling around
                    
                
                
                    
                        dr|z3d
                    
                    
                        I did mentioned something to you in private yesterday re bans, probably got lost in the noise.
                    
                
                
                    
                        dr|z3d
                    
                    
                        *mention
                    
                
                
                    
                        dr|z3d
                    
                    
                        I've caught orignal's toping borg.
                    
                
                
                    
                        zzz
                    
                    
                        about the ban time?
                    
                
                
                    
                        dr|z3d
                    
                    
                        no, no, about the number of bans.
                    
                
                
                    
                        dr|z3d
                    
                    
                        unrelated, updated the maketorblocklist script, not it does the work for you, pulls the list, modifies, saves, deletes downloaded file.  git.skank.i2p/i2pplus/I2P.Plus/src/commit/458fbec9662e897dfbeb4f05fcd8d1e9939480ea/installer/resources/maketorblocklist.sh    
                    
                
                
                    
                        dr|z3d
                    
                    
                        *now
                    
                
                
                    
                        zzz
                    
                    
                        yup, nice job, if we ever need it I'm ready, but for now I don't see a need to be proactive about it
                    
                
                
                    
                        zzz
                    
                    
                        irc is choppy, if I missed something in DM, try again please
                    
                
                
                    
                        orignal
                    
                    
                        <Vort> интересно. количество флудфилов в базе уменьшилось (до 1.5к) и при этом поползло вверх количество транспортов (сейчас по 5к NTCP2 и SSU2, раньше было по 3к)
                    
                
                
                    
                        orignal
                    
                    
                        Vort's report
                    
                
                
                    
                        orignal
                    
                    
                        less floodfills more transports
                    
                
                
                    
                        dr|z3d
                    
                    
                        transports or transits?
                    
                
                
                    
                        orignal
                    
                    
                        NTCP2 and SSU2 sessions
                    
                
                
                    
                        dr|z3d
                    
                    
                        lots of very transient connections it appears.
                    
                
                
                    
                        orignal
                    
                    
                        zzz asked to report obrevations
                    
                
                
                    
                        dr|z3d
                    
                    
                        these are swamping my logs right now: Signature: ECDSA_SHA256_P256 with SendStreamID but no connection found ➜ Recently disconnected
                    
                
                
                    
                        zzz
                    
                    
                        thanks orignal/Vort
                    
                
                
                    
                        dr|z3d
                    
                    
                        nothing important, zzz, was just remarking on the # bans for the class of router we're tracking being lower than expected.
                    
                
                
                    
                        dr|z3d
                    
                    
                        (yesterday)
                    
                
                
                    
                        zzz
                    
                    
                        ok
                    
                
                
                    
                        zzz
                    
                    
                        the bigly update should help also
                    
                
                
                    
                        dr|z3d
                    
                    
                        these disconnections are continuing apace, looks like a possible new strategy. establish connection, drop connection. 
                    
                
                
                    
                        dr|z3d
                    
                    
                        either that or the entire floodnet is being taken down.
                    
                
                
                    
                        zzz
                    
                    
                        yeah high levels of incoming conn esp SSU2, our throttler is working well, haven't seen need to tune it yet
                    
                
                
                    
                        zzz
                    
                    
                        04/29 14:14:31.084 WARN  [ handler 1/1] sport.udp.EstablishmentManager: Probabalistic drop incoming (p=26/128) last rate 131/min current rate 144
                     
                
                
                    
                        orignal
                    
                    
                        Vort also sees mammoth's shit
                    
                
                
                    
                        orignal
                    
                    
                        let's define the threshold we don't accept older RI
                    
                
                
                    
                        orignal
                    
                    
                        72 hours?
                    
                
                
                    
                        zzz
                    
                    
                        yeah I wanted to discuss that
                    
                
                
                    
                        zzz
                    
                    
                        not sure but the old ones may be coming from i2pd, I;m starting to research
                    
                
                
                    
                        zzz
                    
                    
                        what are your current limits?
                    
                
                
                    
                        orignal
                    
                    
                        I drop only future RIs
                    
                
                
                    
                        orignal
                    
                    
                        then delete older druing cleanup
                    
                
                
                    
                        zzz
                    
                    
                        hmm
                    
                
                
                    
                        zzz
                    
                    
                        looking...
                    
                
                
                    
                        dr|z3d
                    
                    
                        we drop them in RefreshRoutersJob, but not solely there.
                    
                
                
                    
                        orignal
                    
                    
                        we need to set a limit
                    
                
                
                    
                        orignal
                    
                    
                        I guess 12 hours is fair enough
                    
                
                
                    
                        zzz
                    
                    
                        we have always had limits. Yes I recommend you add some
                    
                
                
                    
                        zzz
                    
                    
                        here's ours:
                    
                
                
                    
                        zzz
                    
                    
                        assuming we have been up > 1 hour:
                    
                
                
                    
                        orignal
                    
                    
                        yes I will add
                    
                
                
                    
                        zzz
                    
                    
                        if we are ff: 1 hour
                    
                
                
                    
                        zzz
                    
                    
                        if RI has introducers: 54 minutes
                    
                
                
                    
                        orignal
                    
                    
                        hour about regular RIs?
                    
                
                
                    
                        zzz
                    
                    
                        otherwise, variable depending on size of our netdb: max 27 hours, min 1 hour
                    
                
                
                    
                        orignal
                    
                    
                        why 1 hour for FF?
                    
                
                
                    
                        orignal
                    
                    
                        what if we recieved it as reposnse to our lookup?
                    
                
                
                    
                        orignal
                    
                    
                        it's not a good idea
                    
                
                
                    
                        zzz
                    
                    
                        because routers must publish to you, so no need to keep longer, and they don't have to worry about staying integrated
                    
                
                
                    
                        orignal
                    
                    
                        I'm OBEP
                    
                
                
                    
                        zzz
                    
                    
                        ffs don't have a problem remembering RIs
                    
                
                
                    
                        orignal
                    
                    
                        and a message comes to older IBGW
                    
                
                
                    
                        zzz
                    
                    
                        can always look up again, it's just a way to manage memory usage
                    
                
                
                    
                        orignal
                    
                    
                        yes but what if I recieve older IBGW again?
                    
                
                
                    
                        zzz
                    
                    
                        once we're over 5000 RIs, it's a firm 1 hour expiration. If we receive an old RI we throw it away
                    
                
                
                    
                        orignal
                    
                    
                        even as lookup repoense?
                    
                
                
                    
                        zzz
                    
                    
                        yes, correct
                    
                
                
                    
                        orignal
                    
                    
                        what if IB owner has 200 routers in thier netdb?
                    
                
                
                    
                        orignal
                    
                    
                        they will pick older RI for IBGW
                    
                
                
                    
                        zzz
                    
                    
                        routers try hard to refresh their RIs, especially in the first hour after startup
                    
                
                
                    
                        zzz
                    
                    
                        old RIs usually still work
                    
                
                
                    
                        dr|z3d
                    
                    
                        well this is curious: [SSU2] Received SessionRequest from invalid address/port: 176.9.63.240:9050
                    
                
                
                    
                        zzz
                    
                    
                        on one router right now I have 4400 RIs, oldest is < 4 hours
                    
                
                
                    
                        zzz
                    
                    
                        9050 is tor
                    
                
                
                    
                        dr|z3d
                    
                    
                        exactly.
                    
                
                
                    
                        zzz
                    
                    
                        not on your list, might be just bad luck
                    
                
                
                    
                        orignal
                    
                    
                        let's limit by 12 hours for now
                    
                
                
                    
                        not_bob
                    
                    
                        I'm still getting crap for build success on many of my routers.
                    
                
                
                    
                        zzz
                    
                    
                        up to you orignal but that sounds fine... don't forget that reseed RIs may be older than that
                    
                
                
                    
                        not_bob
                    
                    
                        And, it appears that many of my routers don't have a working http tunnel for whatever reason.
                    
                
                
                    
                        dr|z3d
                    
                    
                        you updated + yet not_bob?
                    
                
                
                    
                        not_bob
                    
                    
                        dr|z3d: Everything.
                    
                
                
                    
                        dr|z3d
                    
                    
                        when did you update +?
                    
                
                
                    
                        not_bob
                    
                    
                        less than 12 hours ago again.
                    
                
                
                
                    
                        dr|z3d
                    
                    
                        ok, should be looking satisfactory then.
                    
                
                
                    
                        not_bob
                    
                    
                        Looks like my i2p routers are not reporting as often as they should.
                    
                
                
                    
                        not_bob
                    
                    
                        But, the ones that are have improved success rates.
                    
                
                
                    
                        not_bob
                    
                    
                        Let me make some adjustments.
                    
                
                
                    
                        not_bob
                    
                    
                        It could be on the I2P+ end.  That would explain all the failures.
                    
                
                
                    
                        zzz
                    
                    
                        dr|z3d, are you helping out postman? had some luck sending email yesterday, not so much today
                    
                
                
                    
                        dr|z3d
                    
                    
                        zzz: yeah, keeping postie abreast of the changes.
                    
                
                
                    
                        zzz
                    
                    
                        super, thanks
                    
                
                
                    
                        not_bob
                    
                    
                        I increased the tunnel count for the b32 that everything gets reported to.  That may make it more stable.
                    
                
                
                    
                        not_bob
                    
                    
                        I'll know in a few hours :)
                    
                
                
                    
                        dr|z3d
                    
                    
                        zero toleration for invalid address/port here.
                    
                
                
                    
                        not_bob
                    
                    
                        I'll check back in a number of hours.  Later all!
                    
                
                
                    
                        orignal
                    
                    
                        zzz and we don't flood older than 1 hour. right?
                    
                
                
                    
                        zzz
                    
                    
                        orignal, I don't think we have different rules for flooding, let me check
                    
                
                
                    
                        zzz
                    
                    
                        yeah since the limit is 1 hour if ff, we won't store or flood if older
                    
                
                
                    
                        zzz
                    
                    
                        so that sounds right
                    
                
                
                    
                        orignal
                    
                    
                        Tunnel creation success rate: 17%
                    
                
                
                    
                        orignal
                    
                    
                        not bad
                    
                
                
                    
                        dr|z3d
                    
                    
                        ~70-80% here
                    
                
                
                    
                        dr|z3d
                    
                    
                        ok, since we're battening down the hatches around here, what about addressing this excessive introducers issue?
                    
                
                
                    
                        dr|z3d
                    
                    
                        worth setting a hard limit?
                    
                
                
                    
                        orignal
                    
                    
                        what's with it?
                    
                
                
                    
                        orignal
                    
                    
                        I have fixed that bug long time ago
                    
                
                
                    
                        dr|z3d
                    
                    
                        seems suspect when you've got 10+ introducers configured.
                    
                
                
                    
                        dr|z3d
                    
                    
                        really, anything > 3 seems suspect to me.
                    
                
                
                    
                        orignal
                    
                    
                        maybe old code?
                    
                
                
                    
                        orignal
                    
                    
                        we had the bug about it
                    
                
                
                    
                        dr|z3d
                    
                    
                        could be.. you're setting a hard limit now?
                    
                
                
                    
                        orignal
                    
                    
                        I publish only 3
                    
                
                
                    
                        dr|z3d
                    
                    
                        ok, good.
                    
                
                
                    
                        orignal
                    
                    
                        for receiving not sure what's allowed. I would limited to 10, e.g 0-9
                    
                
                
                    
                        dr|z3d
                    
                    
                        I'd be inclined to ban anything with > 3 tbh.
                    
                
                
                    
                        orignal
                    
                    
                        I wuld do it for 9+
                    
                
                
                    
                        dr|z3d
                    
                    
                        whatever we chose, probably not a bad idea to firm up the spec.
                    
                
                
                    
                        dr|z3d
                    
                    
                        have you got a minute, zzz? Trying to work out why I'm seeing log entries that repeatedly report a ban for an IP that should already be banned.
                    
                
                
                    
                        dr|z3d
                    
                    
                        it's the invalid 9050 offender.
                    
                
                
                    
                        zzz
                    
                    
                        perhaps the log is for a port check that is before the ban check?
                    
                
                
                    
                        dr|z3d
                    
                    
                        oh, could be. do I need to reverse these? if (!TransportUtil.isValidPort(from.getPort()) || !_transport.isValid(fromIP))
                     
                
                
                    
                        zzz
                    
                    
                        not necessarily
                    
                
                
                    
                        zzz
                    
                    
                        generally speaking you should do the most efficient check first
                    
                
                
                    
                        zzz
                    
                    
                        and keep in mind the order of checks when eyeballing logs
                    
                
                
                    
                        dr|z3d
                    
                    
                        hmm
                    
                
                
                    
                        dr|z3d
                    
                    
                        here's the relevant block, I can't see any issue with it, maybe you can?
                    
                
                
                    
                        dr|z3d
                    
                    
                                byte[] fromIP = from.getIP();
                    
                
                
                    
                        dr|z3d
                    
                    
                                String ipAddress = Addresses.toString(fromIP);
                    
                
                
                    
                        dr|z3d
                    
                    
                                boolean hasIP = fromIP != null;
                    
                
                
                    
                        dr|z3d
                    
                    
                                boolean isBlocklisted = hasIP && _context.blocklist().isBlocklisted(fromIP);
                     
                
                
                    
                        dr|z3d
                    
                    
                                if (!TransportUtil.isValidPort(from.getPort()) || !_transport.isValid(fromIP)) {
                     
                
                
                    
                        dr|z3d
                    
                    
                                    if (_log.shouldWarn()) {
                    
                
                
                    
                        dr|z3d
                    
                    
                                        _log.warn("[SSU2] Received SessionRequest from invalid address/port: " + from);
                    
                
                
                    
                        dr|z3d
                    
                    
                                    }
                    
                
                
                    
                        dr|z3d
                    
                    
                                    if (!isBlocklisted) {
                    
                
                
                    
                        dr|z3d
                    
                    
                                       _context.blocklist().add(fromIP, "Invalid address/port in SessionRequest");
                     
                
                
                    
                        dr|z3d
                    
                    
                                       if (_log.shouldWarn()) {
                    
                
                
                    
                        dr|z3d
                    
                    
                                            _log.warn("[SSU2] Banning " + ipAddress + " for duration of session -> Invalid address/port in SessionRequest");
                    
                
                
                    
                        dr|z3d
                    
                    
                                       }
                    
                
                
                    
                        dr|z3d
                    
                    
                                    } else if (isBlocklisted) {
                    
                
                
                    
                        dr|z3d
                    
                    
                                        if (_log.shouldInfo()) {
                    
                
                
                    
                        dr|z3d
                    
                    
                                            _log.info("[SSU2] Not banning " + ipAddress + " -> Already in blocklist");
                    
                
                
                    
                        dr|z3d
                    
                    
                                        }
                    
                
                
                    
                        dr|z3d
                    
                    
                                    }
                    
                
                
                    
                        dr|z3d
                    
                    
                                    return;
                    
                
                
                    
                        dr|z3d
                    
                    
                                }
                    
                
                
                    
                        zzz
                    
                    
                        well, on its face, isBlocklisted will be false if fromIP == null
                    
                
                
                    
                        dr|z3d
                    
                    
                        yeah, except fromIP should never be null, hasIP is probably not required.
                    
                
                
                    
                        dr|z3d
                    
                    
                        if it was null, then ipAddress would return null or error, which it never does.
                    
                
                
                    
                        zzz
                    
                    
                        as I've repeatedly cautioned, your strategy of banning/blocking for every transgression is problematic, especially in the current climate of widespread spoofed/fake RIs; don't get carried away
                    
                
                
                    
                        dr|z3d
                    
                    
                        *nods and smiles*
                    
                
                
                    
                        dr|z3d
                    
                    
                        you can spoof an ip, but you can't spoof a direction connection from that ip, which is what I'm targeting here.
                    
                
                
                    
                        dr|z3d
                    
                    
                        *direct
                    
                
                
                    
                        dr|z3d
                    
                    
                        unless I'm missing something.
                    
                
                
                    
                        zzz
                    
                    
                        you can with UDP if it's pre-token
                    
                
                
                    
                        dr|z3d
                    
                    
                        ok, noted, thanks.
                    
                
                
                    
                        dr|z3d
                    
                    
                        so let's see if we can extract a router hash instead, I guess.
                    
                
                
                    
                        zzz
                    
                    
                        be esp. cautious of IP blocks, because there's no expiratoin
                    
                
                
                    
                        zzz
                    
                    
                        although it is LRU
                    
                
                
                    
                        zzz
                    
                    
                        which reminds me, eyedeekay -
                    
                
                
                    
                        zzz
                    
                    
                        what in the world is the PROP_BLOCKLIST_EXPIRE_INTERVAL stuff you added last summer?
                    
                
                
                    
                        zzz
                    
                    
                        it's a little messy
                    
                
                
                    
                        orignal
                    
                    
                        shouldn't we ban LRU routers?
                    
                
                
                    
                        zzz
                    
                    
                        sorry, LRU = least recently used cache
                    
                
                
                    
                        orignal
                    
                    
                        I though router caps ))
                    
                
                
                    
                        zzz
                    
                    
                        ))
                    
                
                
                    
                        zzz
                    
                    
                        eyedeekay, an int # of ms overflows a lot faster than you think, among other problems....
                    
                
                
                    
                        orignal
                    
                    
                        btw what do we do with such routers?
                    
                
                
                    
                        orignal
                    
                    
                        with U and R together
                    
                
                
                    
                        dr|z3d
                    
                    
                        yeah, I was thinking about that yesterday, no time limit for ip blocks.
                    
                
                
                    
                        dr|z3d
                    
                    
                        U + R caps, if routers exist publishing both, should be instant ban, no? 
                    
                
                
                    
                        dr|z3d
                    
                    
                        that has to be modded code, surely. normal routers won't publish both.
                    
                
                
                    
                        eyedeekay
                    
                    
                        It seems like it would be mutually exclusive
                    
                
                
                    
                        dr|z3d
                    
                    
                        have you actually seen any routers like that, orignal?
                    
                
                
                    
                        dr|z3d
                    
                    
                        should be, eyedeekay, but there are some nuts out there messing with code :)
                    
                
                
                    
                        dr|z3d
                    
                    
                        I'm staring at about -8000 bans right now vs yesterday.
                    
                
                
                    
                        dr|z3d
                    
                    
                        6-8K, something like that, rough estimate.
                    
                
                
                    
                        eyedeekay
                    
                    
                        Re: expire bans, added it b/c of the 'shitty bans' as orignal put it at the time as an option to expire all bans at a user-specified time, but it's probably also an example of obscuratus and I having similar blindspots
                    
                
                
                    
                        eyedeekay
                    
                    
                        Nobody uses it and it's probably not very useful anyway, and the key source of said bans went away last checkin, so I could just revert it now
                    
                
                
                    
                        zzz
                    
                    
                        yes eyedeekay I recommend reverting. It looks like an experiment, not a feature, and not very useful either
                    
                
                
                    
                        zzz
                    
                    
                        - can't change the time after startup
                    
                
                
                    
                        zzz
                    
                    
                        - cache is LRU already
                    
                
                
                    
                        zzz
                    
                    
                        - Integer instead of int
                    
                
                
                    
                        zzz
                    
                    
                        - Integer instead of long, overflow will be very bad, infinite loop
                    
                
                
                    
                        dr|z3d
                    
                    
                        low hanging fruit.
                    
                
                
                    
                        orignal
                    
                    
                        does Java do it now?
                    
                
                
                    
                        orignal
                    
                    
                        or it needs to be implemented?
                    
                
                
                    
                        dr|z3d
                    
                    
                        probably needs implementing, not difficult.
                    
                
                
                    
                        dr|z3d
                    
                    
                        and done.
                    
                
                
                    
                        orignal
                    
                    
                        yes, that's what I mean
                    
                
                
                    
                        dr|z3d
                    
                    
                                boolean reachable = cap != null && cap.indexOf(Router.CAPABILITY_REACHABLE) >= 0;
                     
                
                
                    
                        dr|z3d
                    
                    
                                boolean unreachble = cap != null && cap.indexOf(Router.CAPABILITY_UNREACHABLE) >= 0;
                     
                
                
                    
                        dr|z3d
                    
                    
                                if (reachable && unreachable) {
                    
                
                
                    
                        dr|z3d
                    
                    
                                    _context.banlist().banlistRouter(h, " <b>➜</b> Invalid published capabilities (RU)", null,
                     
                
                
                    
                        dr|z3d
                    
                    
                                                                     null, _context.clock().now() + 24*60*60*1000);
                    
                
                
                    
                        dr|z3d
                    
                    
                                    _msg3p2FailReason = NTCPConnection.REASON_BANNED;
                    
                
                
                    
                        dr|z3d
                    
                    
                                    if (_log.shouldWarn() && !isBanned)
                    
                
                
                    
                        dr|z3d
                    
                    
                                        _log.warn("Banning for 24h and immediately disconnecting from Router [" + h.toBase64().substring(0,6) + "]" +
                    
                
                
                    
                        dr|z3d
                    
                    
                                                  " -> Publishing both R and U caps");
                    
                
                
                    
                        dr|z3d
                    
                    
                                    _context.simpleTimer2().addEvent(new Disconnector(h), 3*1000);
                     
                
                
                    
                        dr|z3d
                    
                    
                                    throw new DataFormatException("Invalid caps (RU): " + h);
                     
                
                
                    
                        dr|z3d
                    
                    
                                }
                    
                
                
                    
                        dr|z3d
                    
                    
                        * boolean unreachable