+R4SAS
                        
                        
                            +RN
                        
                        
                            +RN_
                        
                        
                            +T3s|4
                        
                        
                            +Xeha
                        
                        
                            +acetone
                        
                        
                            +not_bob_afk
                        
                        
                            +orignal
                        
                        
                            Irc2PGuest33667
                        
                        
                            Onn4l7h
                        
                        
                            T3s|4_
                        
                        
                            aargh2
                        
                        
                            b4dab00m
                        
                        
                            cumlord
                        
                        
                            leopold
                        
                        
                            mareki2p_
                        
                        
                            profetik1
                        
                        
                            shiver_
                        
                        
                            u5657
                        
                        
                            vvx2
                        
                        
                            x74a6
                        
                    
                    
                        orignal
                    
                    
                        good idea
                    
                
                
                    
                        orignal
                    
                    
                        will do the same
                    
                
                
                    
                        zzz
                    
                    
                        here's where I sent a message to i2pd, retransmitted 3x, gave up after 10 seconds, then got acks for all of them 1 1/2 seconds later:
                    
                
                
                    
                        zzz
                    
                    
                        07-09 13:36:19.723 WARN  [acket pusher] outer.transport.udp.PeerState2: Message expired: OB Message 2046472889 seq 84 type 23 size 2122 fragments: 2 volleys: 4 lifetime: 10044 unacked fragments: 0 sizes: 1417 705 send counts: 4 1  to: [2001:470:1f12:e56:0:0:0:2]:24768 BpATV4 OB2  recvAge: 10s sendAge: 10s sendAttemptAge: 3025ms sendACKAge: 10s lifetime: 117m RTT: 17 RTO: 8000 MTU: 1308 LMTU: 1500 cwin: 1500 acwin: 5137 
                      
                
                
                    
                        zzz
                    
                    
                        SST: 2744 FRTX? false consecFail: 1 msgs rcvd: 6 msgs sent: 85 pkts rcvd OK/Dup: 95/0 pkts sent OK/Dup: 91/4 IBM: 0 OBQ: 0 OBL: 0 theyRelayToUsAs: 3696531151
                    
                
                
                    
                        zzz
                    
                    
                        07-09 13:36:21.246 WARN  [ handler 1/1] outer.transport.udp.PeerState2: Acked but not found in outbound messages: OB Message 2046472889 seq 84 type 23 size 2122 fragments: 2 volleys: 4 lifetime: 11567
                     
                
                
                    
                        zzz
                    
                    
                        07-09 13:36:21.246 WARN  [ handler 1/1] outer.transport.udp.PeerState2: Dup ACK of fragment 0 of OB Message 2046472889 seq 84 type 23 size 2122 fragments: 2 volleys: 4 lifetime: 11567
                     
                
                
                    
                        zzz
                    
                    
                        07-09 13:36:21.246 WARN  [ handler 1/1] outer.transport.udp.PeerState2: Dup ACK of fragment 0 of OB Message 2046472889 seq 84 type 23 size 2122 fragments: 2 volleys: 4 lifetime: 11567
                     
                
                
                    
                        zzz
                    
                    
                        07-09 13:36:21.246 WARN  [ handler 1/1] outer.transport.udp.PeerState2: Dup ACK of fragment 0 of OB Message 2046472889 seq 84 type 23 size 2122 fragments: 2 volleys: 4 lifetime: 11567
                     
                
                
                    
                        zzz
                    
                    
                        07-09 13:36:21.246 WARN  [ handler 1/1] outer.transport.udp.PeerState2: Dup ACK of fragment 0 of OB Message 2046472889 seq 84 type 23 size 2122 fragments: 2 volleys: 4 lifetime: 11567
                     
                
                
                    
                        orignal
                    
                    
                        so what's your point?
                    
                
                
                    
                        orignal
                    
                    
                        you think I miss packets?
                    
                
                
                    
                        zzz
                    
                    
                        no. you got all the packets but never sent an ack
                    
                
                
                    
                        zzz
                    
                    
                        until 11.5 seconds later
                    
                
                
                    
                        orignal
                    
                    
                        what was after 11.5 secs?
                    
                
                
                    
                        orignal
                    
                    
                        Just ack?
                    
                
                
                    
                        orignal
                    
                    
                        3 different acks?
                    
                
                
                    
                        zzz
                    
                    
                        don't know, not logged
                    
                
                
                    
                        orignal
                    
                    
                        or Ack with I2NP block?
                    
                
                
                    
                        orignal
                    
                    
                        my understanding
                    
                
                
                    
                        orignal
                    
                    
                        I have sent Acks but they didn't reach you
                    
                
                
                    
                        orignal
                    
                    
                        after 11.5 I tried to send a new message 
                    
                
                
                    
                        orignal
                    
                    
                        with Ack block
                    
                
                
                    
                        zzz
                    
                    
                        maybe
                    
                
                
                    
                        orignal
                    
                    
                        then how can I fix it?
                    
                
                
                    
                        zzz
                    
                    
                        just mentioning, in case you have a bug where you're not sending the acks.
                    
                
                
                    
                        zzz
                    
                    
                        I will look for more delayed acks and see if there's a pattern
                    
                
                
                    
                        zzz
                    
                    
                        you send an ack of the new packet even if the I2NP message inside is a duplicate, right???
                    
                
                
                    
                        orignal
                    
                    
                        no
                    
                
                
                    
                        orignal
                    
                    
                        I only send ack for a packet I didn't receive before
                    
                
                
                    
                        orignal
                    
                    
                        and if a packet contains I2NP or fragments
                    
                
                
                    
                        zzz
                    
                    
                        the distinction is between 'packet I didn't receive before' and 'packet containing i2np msg or fragment I didn't receive before'
                    
                
                
                    
                        zzz
                    
                    
                        sounds like you do it right
                    
                
                
                    
                        zzz
                    
                    
                        will keep looking
                    
                
                
                    
                        orignal
                    
                    
                        let me change it
                    
                
                
                    
                        zzz
                    
                    
                        so you don't do it right?
                    
                
                
                    
                        orignal
                    
                    
                        myabe I should send Ack again if I receive duplicate
                    
                
                
                    
                        orignal
                    
                    
                        need to check SSU1 code
                    
                
                
                    
                        zzz
                    
                    
                        example:
                    
                
                
                    
                        zzz
                    
                    
                        I send packet 1 with i2np message 12345
                    
                
                
                    
                        zzz
                    
                    
                        you send ACK 1
                    
                
                
                    
                        zzz
                    
                    
                        ACK 1 is lost
                    
                
                
                    
                        zzz
                    
                    
                        I send packet 2 with i2np message 12345
                    
                
                
                    
                        zzz
                    
                    
                        do you send ACK 2 or not?
                    
                
                
                    
                        orignal
                    
                    
                        and I don't send ack
                    
                
                
                    
                        orignal
                    
                    
                        wait
                    
                
                
                    
                        orignal
                    
                    
                        if i's packet 2 I send ack
                    
                
                
                    
                        zzz
                    
                    
                        ok good, you do it right
                    
                
                
                    
                        orignal
                    
                    
                        but there is something else I don't do right
                    
                
                
                    
                        orignal
                    
                    
                        I receive packet 1 and handle it
                    
                
                
                    
                        orignal
                    
                    
                        I receive packet 2 and also handle it
                    
                
                
                    
                        orignal
                    
                    
                        with the same msg 12345
                    
                
                
                    
                        R4SAS
                    
                    
                        BpAT ... is mine
                    
                
                
                    
                        R4SAS
                    
                    
                        rebuilt 2 days ago
                    
                
                
                    
                        zzz
                    
                    
                        ok. is the internet reliable for it? It seems to have more errors than most routers
                    
                
                
                    
                        R4SAS
                    
                    
                        ofcourse reliable, this is dedicated server
                    
                
                
                    
                        zzz
                    
                    
                        ok, will keep an eye on it
                    
                
                
                    
                        zzz
                    
                    
                        good to know
                    
                
                
                    
                        R4SAS
                    
                    
                        ff + transit
                    
                
                
                    
                        R4SAS
                    
                    
                           Routers: 5393 Floodfills: 1615 LeaseSets: 138
                    
                
                
                    
                        R4SAS
                    
                    
                           Client Tunnels: 290 Transit Tunnels: 1952
                    
                
                
                    
                        zzz
                    
                    
                        definitely seeing a lot of late acks from i2pd, after I've already timed out
                    
                
                
                    
                        zzz
                    
                    
                        including 2RRY
                    
                
                
                    
                        zzz
                    
                    
                        I have timestamps if you have the logs for it orignal 
                    
                
                
                    
                        zzz
                    
                    
                        R4SAS, do you have logs on BpAT for individual packets and acks?
                    
                
                
                    
                        orignal
                    
                    
                        no
                    
                
                
                    
                        orignal
                    
                    
                        but I will check
                    
                
                
                    
                        orignal
                    
                    
                        late acks for how long?
                    
                
                
                    
                        orignal
                    
                    
                        and 2RRY is not a good example
                    
                
                
                    
                        zzz
                    
                    
                        as I said above, after my 10 second timeout
                    
                
                
                    
                        orignal
                    
                    
                        as you know it really has netwrok issues
                    
                
                
                    
                        orignal
                    
                    
                        anyway I will take a look what might cause it
                    
                
                
                    
                        R4SAS
                    
                    
                        zzz: i have logs disabled, transit traffic generates too much messages\
                    
                
                
                    
                        R4SAS
                    
                    
                        but i can enable them
                    
                
                
                    
                        zzz
                    
                    
                        maybe a lower-bandwidth router would be better
                    
                
                
                    
                        zzz
                    
                    
                        here's the router I got late acks from in last 12 hours: ImQC kyY2 2RRY YXEA BpAT = all i2pd
                    
                
                
                    
                        R4SAS
                    
                    
                        I have ssu2-only router if you interested
                    
                
                
                    
                        orignal
                    
                    
                        all R4SAS's ))
                    
                
                
                    
                        orignal
                    
                    
                        but 2RRY
                    
                
                
                    
                        R4SAS
                    
                    
                        нгз
                    
                
                
                    
                        R4SAS
                    
                    
                        yup
                    
                
                
                    
                        R4SAS
                    
                    
                        )))
                    
                
                
                    
                        zzz
                    
                    
                        any of those low-bandwidth?
                    
                
                
                    
                        R4SAS
                    
                    
                        none
                    
                
                
                    
                        R4SAS
                    
                    
                        but I can disable transit
                    
                
                
                    
                        R4SAS
                    
                    
                        lets do this on kyY2
                    
                
                
                    
                        zzz
                    
                    
                        we need some router with low-level logging on
                    
                
                
                    
                        R4SAS
                    
                    
                        disabling transit now
                    
                
                
                    
                        R4SAS
                    
                    
                        I'll enable logging in 15 minutes
                    
                
                
                    
                        zzz
                    
                    
                        maybe you introduced a bug recently? or maybe you two are running all the high-bandwidth routers? wondering why all the routers doing this are yours...
                    
                
                
                    
                        zzz
                    
                    
                        also, we really need the termination issues fixed so we can reduce the number of packet failures
                    
                
                
                    
                        R4SAS
                    
                    
                        most of them are high-bandwidth
                    
                
                
                    
                        R4SAS
                    
                    
                        I'll rebuild with new commits today
                    
                
                
                    
                        orignal
                    
                    
                        my that home ipv6 is low bndwidth
                    
                
                
                    
                        orignal
                    
                    
                        zzz I have more routers
                    
                
                
                    
                        orignal
                    
                    
                        that one with family=gostcoin and I rebuilt it yesteraday
                    
                
                
                    
                        orignal
                    
                    
                        I'm working on termination issue now
                    
                
                
                    
                        zzz
                    
                    
                        ok. missing termination and late acks are the two causes of message loss I'm seeing right now
                    
                
                
                    
                        zzz
                    
                    
                        so many routers R4SAS, maybe you should set a family )))
                    
                
                
                    
                        orignal
                    
                    
                        I will set gotscoin to 2RY
                    
                
                
                    
                        orignal
                    
                    
                        always forget
                    
                
                
                    
                        R4SAS
                    
                    
                        I'm not using family now because it will break ssu2 testing
                    
                
                
                    
                        zzz
                    
                    
                        yeah, it's good for the high bandwidth ones, so we don't put them in the same tunnel
                    
                
                
                    
                        R4SAS
                    
                    
                        maybe will add later
                    
                
                
                    
                        zzz
                    
                    
                        let's focus on getting down to zero lost messages, or as close as we can get
                    
                
                
                    
                        R4SAS
                    
                    
                        stopped kyY2, rebuilding
                    
                
                
                    
                        zzz
                    
                    
                        make sure you're logging lost messages at a high level too so you can track it
                    
                
                
                    
                        orignal
                    
                    
                        his routers have ygg address ending with 333 )))
                    
                
                
                    
                        R4SAS
                    
                    
                        ))
                    
                
                
                    
                        zzz
                    
                    
                        =zzz )
                    
                
                
                    
                        zzz
                    
                    
                        restarting mine too
                    
                
                
                    
                        orignal
                    
                    
                        old joke
                    
                
                
                    
                        orignal
                    
                    
                        and remeber you will be prosecuted for your nick in some "democratic" countries now ))
                    
                
                
                    
                        zzz
                    
                    
                        great
                    
                
                
                    
                        zzz
                    
                    
                        >> teminate session after 5 unacked resends 
                    
                
                
                    
                        zzz
                    
                    
                        is that for one i2np message?
                    
                
                
                    
                        zzz
                    
                    
                        or 5 different i2np messages?
                    
                
                
                    
                        orignal
                    
                    
                        for packets
                    
                
                
                    
                        orignal
                    
                    
                        I resend packets not messages
                    
                
                
                    
                        orignal
                    
                    
                        and this 5 have exponential timeout
                    
                
                
                    
                        zzz
                    
                    
                        that might be a litte drastic? we don't fail the session for one I2NP message, but for 5 consecutive I2NP messages that fail (each one retransmitted 4x)
                    
                
                
                    
                        zzz
                    
                    
                        (same as in SSU 1 for us)
                    
                
                
                    
                        R4SAS
                    
                    
                        started with debugging
                    
                
                
                    
                        zzz
                    
                    
                        ours may not be drastic enough, 5 I2NP may be too high
                    
                
                
                    
                        orignal
                    
                    
                        I will change
                    
                
                
                    
                        orignal
                    
                    
                        also check size of unconrimed packets
                    
                
                
                    
                        orignal
                    
                    
                        btw it might cauased delayed acks
                    
                
                
                    
                        orignal
                    
                    
                        because I don't change Ack block when resend
                    
                
                
                    
                        orignal
                    
                    
                        ofc it's 5 packets but "resendable"
                    
                
                
                    
                        orignal
                    
                    
                        e.g. with I2NP message inside
                    
                
                
                    
                        orignal
                    
                    
                        *5 resends