IRCaBot 2.1.0
GPLv3 © acetone, 2021-2022
#ls2
/2022/07/11
@eyedeekay
+R4SAS
+RN
+RN_
+T3s|4
+Xeha
+acetone
+orignal
+weko
Irc2PGuest58867
Leopold_
Onn4l7h
Onn4|7h
T3s|4_
aargh2
anon2
cancername
eyedeekay_bnc
hk
mareki2p
not_bob_afk
profetikla
shiver_
u5657
x74a6
orignal but I don't know how to fix
orignal if NACKS > 255
orignal that what happens
orignal add range 255 0?
zzz yes
zzz but that didn't happen here
orignal I know what happened there
orignal now I'm asking about more generic case
orignal to avoid zero acnt in future
orignal want to rewrite that code
zzz yup
orignal I don't produces ranges 255 0 and 0 255 yet
orignal fixing it now
orignal fixed
zzz happened again, 10 minutes ago, with 2RRY - no ack of session confirmed, no ack of any packets
orignal I haven't fixed for SessionConfirmed yet
orignal SSU2: Unexpected message type 168 from [2a02:180:2:92:92:92:11:15]:16799
orignal looks like it was mass outage on ipv6
zzz got some fixes to do on my side too
orignal do you know is there are similar issue for SSU1?
orignal because I use the same sending code
zzz not sure
orignal I mean if I sent a datagram and there was not error
orignal what else I can do?
zzz if the internet drops everything for 10 seconds, there's not a lot to do
orignal not sure if it was everything or just UDP
orignal coz send buffer overflow
orignal maybe we can check it and queue it up, etc.
orignal I will add the code to check actual sent size
zzz count of session confirmed retransmissions by router, last 24 hours, highest ones:
zzz 67 tXc6dH
zzz 49 k8vhnd
zzz 49 BpATV4
zzz 41 kyY2Tx
zzz 28 3PYq0v
zzz 19 YXEAXl
zzz 18 bAU~6X
zzz 16 CEFnjX
zzz 14 xZ9nsA
zzz 12 2RRYXk
zzz 9 ImQCa~
zzz all i2pd
orignal I know because it's not implemneted
orignal will fix today
orignal btw, you don't receive more SessionCreated from me
zzz huh?
orignal you restransmit sessionconfrimed
zzz right
zzz just typing out the scenarios for my own sanity:
orignal I'm wordiring if I have received it
orignal if I didn't I would keep seinding sessioncreated
zzz when I'm retransmitting session confirmed, I'm also retransmitting one or more i2np messages
zzz so there's two cases:
zzz 1) you didn't get the first session confirmed. when you get the retx, send ack 0, and decrypt the data packets if you queued them, and ack them
zzz 2) you did get the first session confirmed. you won't be able to retransmit the retransmitted session confirmed,
zzz *you won't be able to decrypt the retransmitted session confirmed,
zzz but you will be able to decrypt the retransmitted data packets, so send ACK 2-0, 3-0, 4-0, or whatever for each data packet you get
zzz that will ack both the session confirmed and the data packets
zzz so both scenarios will result in bob sending an ack each time he gets something
zzz so you shouldn't need to resend ack 0 if you get 'garbage' (retransmitted session confirmed). You can if you want I guess (I don't)
orignal I will fix 2
orignal and send ack again
orignal but my question is why whole communication gets stuck
orignal so the problem is that I don't send ack even to data packet
zzz right. no idea why. I keep sending stuff, never get anything back
zzz this is the one problem with XK that's a little tricky to handle. IK doesn't have this issue
orignal and you are saying that you never send sessionconfirmed retransmission to Java routers?
zzz not saying that. The vast majority and all the top ones are to i2pd. Don't have an exact count but about 335 out of 350 total retransmissions in last 24 hours were to i2pd
orignal maybe because it was a spike in SSU1?
orignal since I don't control flow there
orignal it might be flood of UDP packets
zzz no ideas here, sorry
orignal I don't see R4SAS's SSU2 only routers here
orignal that's why
orignal I meant in your list
zzz here's the rest of the list, fyi:
zzz 5 OANQwi
zzz 5 nYlJtl
zzz 4 Qreu1M
zzz 4 PeGQYG
zzz 4 p~8-FO
zzz 4 MNcWgU
zzz 4 jGiQZ8
zzz 4 iNmqNX
zzz 4 gpUBQf
zzz 2 -Ym2vN
orignal his is gpUBQf
zzz yup
orignal maybe that's the reason
zzz if you're overflowing some queue you would log it, right?
orignal how is it possible?
orignal you send an UDP packet and system or routers just drop it
zzz I'm not going to make guesses about your code
orignal for example at works we use different VLANs for multicast
orignal on different nteowrk cards
orignal it's not about my code
orignal it's generic UDP
orignal no delays and no error even if a packet gets dropped
zzz but this is over at least 10 seconds. I'm sending the data packet 4 times at 0, 1, 3, 7 seconds and timing out at 10
zzz and never get an ack
R4SAS zzz: can you print all incoming packets to socket?
R4SAS orignal: and you all sending
zzz so I've sent session confirmed 4 or 5 times (packet 0) and (at least) data packets 1-4
zlatinb testnet + wireshark time :)
zzz R4SAS, see scrollback, 16:14 eastern (UTC - 4) yesterday, for an example
R4SAS zzz: this is parsed messages
R4SAS I'm about raw packets with only "SRCIP, seq N, DSTIP"
zzz not necessary yet. I'm waiting to hear if i2pd _did_ get the 4 data packets and _did_ send an ack for each
R4SAS same thing I proposing for orignal but for outgoing packets
R4SAS to see if ACKs really sent
R4SAS zlatinb: if u can, that would help
zlatinb when my hand heals I can look into it
zzz it would be tough to reproduce on testnet, because testnet routers like to connect once and stay connected
zzz not the best way to find intermittent handshake issues
orignal there is no such thing as "really sent"
zzz sure
zzz 0) Hi
orignal zzz, i2pd definetly sent acks to data packets
orignal and didn't send to sessionconformed retrans
zzz but I mean in this case where I've given you a timestamp
zzz what's on the agenda for today after a week off?
eyedeekay I have a go-i2p report
orignal SSU2 status
zzz ok go-i2p will be 1), and SSU2 will be 2)
zzz anything else for the list?
zzz ok then
zzz 1) go-i2p
zzz what's the latest?
eyedeekay Beware of copypasta
eyedeekay I am cautiously approaching sending my first SessionRequest message
eyedeekay The KDF documentation was very helpful this week, what I worked on mostly was KDF for handshake message 1, that section of pseudo-code
eyedeekay I need to fit the message-1 KDF stuff I'm working on into the go Noise library I'm tentatively using: github.com/flynn/noise which is what I'm figuring out now-ish
eyedeekay Which I think I can by implementing alternate versions of the library's interfaces rather than needing to do it all
eyedeekay But if that turns out to be outside the capabilities of the library at this time, I'll fork it and see if the upstream is interested in my changes, and if not, I'll need to re-evaluate and pick a different one, maintain a fork, or write my own.
eyedeekay First time is going to be the hardest time with that, once I know how to customize my Noise library adequately doing other protocols will be more like varying a theme
eyedeekay Once I figure out how to do that, I'm pretty sure I can construct a SessionRequest message, connect to to a router, send it, then probably have something on the go-i2p side explode, because it won't know what to do next yet with what it gets back
eyedeekay but if I can read response bytes from Bob after sending the SessionRequest, then I will have officially made a network connection to another I2P router and sent it a message it can hopefully understand, which feels significant
eyedeekay I hope to attempt that by Friday after getting the KDF implemented
eyedeekay With all that I'm going to also push a roadmap for NTCP2 to go-i2p/transports/ntcp/README.md to outline the rest of the stuff that I think will need to be done before it's implementation-complete
eyedeekay Feeling semi-optimistic regarding time, optimistically between 1-2 months for it to reach the Data Phase
eyedeekay There are, I think, some changes I'll need to reach back into the common library to update Router Info's to publish NTCP2 addresses too
eyedeekay It's all a big mess right now with unused versions of functions commented out and notes on why they didn't work, though, I'll have to get rid of most of that before I check anything in
zzz yeah the trick is the mods you'll need for our AES-obfuscation
zzz and padding
eyedeekay Yeah I'm beginning to get that, that's where I think I might need to make changes upstream to flynn/noise
eyedeekay It's a little confusing
eyedeekay But I'm getting my head around it gradually
orignal is it NTCP2?
zzz the way it usually goes when first testing a handshake is that you'll get nothing back because it's not right
zzz so then, maybe, java-side logging will give you a clue
eyedeekay That's also a possibility :)
zzz but in practice, the next step is probably to write both sides (alice and bob) for session request, and do a unit test
zzz once that works, then try again vs. java
zzz so, if you're lucky, you'll get something back, but don't count on it
eyedeekay Probably won't but I'm going to give it a try with a router on my LAN as soon as it's a possibility
eyedeekay Interesting either way
zzz sounds like great progress
eyedeekay orignal yes I'm starting with NTCP2, unless I have a good reason otherwise I'm probably not going to do NTCP1
zzz holler if you need help
zzz NTCP1 is definitely dead
eyedeekay Yeah it's finally getting to the point where it's actually going to do something
orignal or you might start witj SSU2
orignal and will be on the same page with us
eyedeekay Right now my immediate hurdle is figuring out how to mod noise
zzz we've been over this many times. SSU2 is at least 3-4x more complex than NTCP2
orignal I even don;t have NTCP1 code in i2pd anymore
eyedeekay Like I said, I probably won't do NTCP1, certainly won't make it a priority
zzz with a unit test you can try it without mods, then add the mods one by one
orignal and you shouldn't do it at all
eyedeekay That's a good point, I can start by testing the mods one-by-one if I test them as I add them
eyedeekay Trying to get things done all-at-once has held me back here before
eyedeekay So that's probably a good idea
zzz but always fun to test in the live net, in the unlikely event that it works, you've avoided a lot of unit testing. It just usually doesn't happen like that
eyedeekay Well either way, learning and progress, that's why I'm doing it
zzz yup
zzz anything else on 1) ?
eyedeekay Not yet
eyedeekay next week
zzz definitely the most progress ever reported, very nice
zzz also, a good thread over on zzz.i2p about use cases
zzz 2) SSU2 status
zzz who wants to go first? I forget whose turn it is
zzz ok, I'll go
zzz found and fixed a nasty bug yesterday where I was treating NACKs as ACKs
zzz my main priority is getting rid of all packet loss (except for normal packet loss)
orignal and found a bug for me
zzz so if I can see a pattern, there's something wrong
zzz I see two main causes right now, both when talking to i2pd:
zzz 1) no acks at all for outbound connections, as discussed above
orignal I think not only outbound
zzz 2) no termination received, I send something after i2pd has timed out
orignal it can happen with any connection
orignal about 2 I have made some changes
zzz I'm continuing to see longer session uptimes, and more traffic, and more routers
orignal you should see more termination blocks
orignal also with reason code
zzz I don't _think_ there's any more relay or peer test issues, but I'm not looking for them right now. I'm focused on making basic data traffic be 100% solid
orignal so, my turn I guess?
zzz EOT, go ahead
orignal let's start with relay and peer test
orignal for me it's higher priority than basic traffic because I can't replace SSU1 by SSU2 without it
orignal after recent bug fixes peer test and relay requests work good
orignal still working on Charlie's side e.g. being firewalled
orignal you mentioned another bug with Acks
orignal that my Ack block always keep growing
orignal because there are gaps in received packed, and some messages never retramsmitted
zzz hmm. you don't "trim" it?
orignal but I need to do it more
orignal also I never sent ranges more than 255 acks or nacks
orignal I need to add few ranges
orignal that I have fixed yesteraday
zzz yeah that one I posted yesterday was 605 all the way down to 0
orignal yes because too many ranges
orignal but it might be possisibility that not too many ranges but each range is big
orignal and I need to clean up older our or sequence messages
orignal I was going to do it today but didn't have time yet
zzz fyi right now my max is 512, and I shift up by 128. So I'll cover at least 384
orignal I was going to start with 1K
orignal or 2K
zzz really depends on max window
orignal e.g. difference between last received seqn and seqn of first gap
zzz but retransmissions are new pkt numbers so it has to be a multiple
orignal yes, that's another issue
zzz the right answer is probably somewhere between 256 and 2K
orignal that's what I'm going to do
orignal the problem is that 2K might come in very short time
orignal less then a second
orignal that's my main concern
zzz sure but doesn't matter how fast, the issue is how big your window can get and how often you're acking
zzz and what the other guy is doing
zzz any other SSU2 status?
orignal so back to issue 1
zzz go ahead
orignal I start suspecting that SSU1 causes it
orignal but need to investigate more
zzz sure, always good to have a theory. then try to prove it true or false
orignal based on my experience if you pump to many UDP packets through the same interface they get dropped
orignal that's why I want to switch to SSU2 insted SSU1 soon
zzz can't switch until it works :)
orignal because SSU1 trafiic is really untrolled
orignal not true
zzz I'll just keep reporting what I see
orignal if you SSU2 doesn't work, SSU1 doesn't work too
orignal I mean even bad working SSU2 is better that SSU1 in i2pd
zzz maybe, maybe not. we don't have any data on packet loss in 1 vs. 2 for i2pd
orignal I kno it's a lot
orignal I didn't fix it assuming we were going to start working on SSU2 anyway
zzz I'm working on changes to temporarily ban routers that have failures like 1) and 2) above
orignal what do you see about 2) now?
zzz I close the session but it's not banned
zzz so we usually just open a new session almost right away
orignal as I said I send termination now
zzz ok I'll look for termination in the logs, please ask everybody to update
zzz any other SSU2 status?
orignal they keeps updating anyway
orignal you should see improvements
orignal at least with my and R4SAS 's routers
orignal another thing is mtu
orignal haven't decided about SSU2 over yggdrasil yet
zzz yes, MTU is part of the "basics"
orignal with 64K MTU
orignal I will implement it soon
orignal but 64K is different story
zzz you should be getting full 1500-byte packets from java routers, at least sometimes
orignal yes I assume it might up to 1500
zzz and never 1501+ :)
zzz anything else for the meeting?
orignal 64K might change the picture dramatically
zzz anybody else have anything more?
zzz ok, I guess that's it, thanks everybody
orignal I would recommend idk to work in parralel on NTCP2 and SSU2
orignal because too much common code and approaches
eyedeekay I get the point of what you're saying there orignal, since zzz did describe SSU2 once to us as NTCP2 over UDP with some QUIC features, but for better or worse I'm going to stick to NTCP2 for now, if there's code I should be sharing between the transports I'll just have to refactor it when I reach that point
orignal eyedeekay for example block
orignal they work same way
orignal handshake and Noise states
orignal if I start implemnting now
orignal I would implemente common code for it
eyedeekay OK now I really do see your point
orignal but NTCP2 was implemneted few years ago
eyedeekay My problem is that I don't fully know which parts would end up common and which parts would end up specific yet
eyedeekay You've given me some clues just now though, block, handshake, and noise states
eyedeekay So I can maybe plan for that
orignal blocks, Noise are common
zzz sure, wouldn't hurt to skim the SSU2 spec to get a sense of what's common
orignal SessionRequest-SessionCreated-SessionConfirmed
zzz they're both XK
orignal Bob doesn't know whohe is talking to because SessionConfirmed
orignal and there he verify S instead signatures
eyedeekay This is good, it seems like since I'm working on modding noise and maybe sending messages now, now is the time to consider where this goes
zzz but I would come at it from the other direction. Not what is common at the low-level (blocks, noise, ...) but what are the common interfaces for a generic transport (send I2NP, receive I2NP, ...) and design your first transport accordingly, so it's not a mess when you add a 2nd one
zzz more succinctly: interface design >>> design for reuse
orignal also think about both ipv4 and ipv6 at the same time
eyedeekay psi stubbed out a transport interface years ago, there is a Transport interface with a sort of "session factory" which makes TransportSessions which are specified read, send, and queue functions
eyedeekay Being slightly agnostic of what I should do, that is part of what I'm working from at the moment
zzz eyedeekay, rereading your status above:
eyedeekay Right now I've made messages a simple interface and made each message an implementation of that interface, SessionRequest is just the one that I've made the most progress on
eyedeekay but I've put them inside of the same module as the NTCP2 transport, if I move them I can easily share that code between transports
eyedeekay I still don't want to do both at the same time but I can maybe plan ahead for some of this now that I know about it
zzz <eyedeekay> I need to fit the message-1 KDF stuff I'm working on into the go Noise library I'm tentatively using: github.com/flynn/noise which is what I'm figuring out now-ish
zzz the NTCP2 spec of message 1 KDF *is* Noise. there shouldn't be anything to "fit in"
eyedeekay Oh well then maybe I've wasted some time
eyedeekay Or maybe I implemented what I thought was different without knowing it...
eyedeekay OMG I made a "spec language" error, this is my mistake
zzz remember this was our first noise protocol. str4d said 'just say follow the noise spec', I wanted to document more so we understood it, and orignal wanted to do really low-level documentation. orignal won
eyedeekay That's pretty funny, well I guess I know a little more about noise now
zzz the AES obfuscation of the eph. keys is really a post-processing / pre-processing step
eyedeekay Ha... talk about doing things the hard way
zzz the one wrinkle is adding the padding, and for messages 2 and 3, is the mixHash(padding)
eyedeekay OK I'm just going to re-read some of these headlines here and take a moment to ask some questions
zzz I'm not sure what you're fitting into what. But glanced at the flynn lib doc and the terminology/naming is identical to the java noise lib we're using
eyedeekay Well I thought I was writing a KDF and implementing it to extend flynn/noise but since I didn't read "as defined in" correctly I think I was just re-writing an existing KDF
eyedeekay apparently I spent a lot of time on something because I was looking at the code block and not the sentence directly above it
eyedeekay Which explains why I couldn't figure out how to fit it in
zzz i2pd didn't use a noise lib, and we all dislike/hate the noise docs, those are the other reasons it's all "unrolled" in our spec
eyedeekay I get it now I think
zzz the "Additions to the Framework" section of the spec should be the full list of changes
eyedeekay So the modifications I would need to do are to fulfill things like the 16-byte option block non-noise payload random padding
zzz they're minor, targeted, specific changes required for our threat model
zzz right
eyedeekay I think that might make some of my problems into non-problems then
zzz everything else, your noise lib should do for you
eyedeekay Well crap. I think I need to regroup and figure out where I'm at, because apparently what I've done is implement a part of noise with a slightly different crypto library than the one that my noise lib used, thinking it was something I had to do
eyedeekay but since flynn/noise has it already then I'm probably not actually behind for it
eyedeekay So many things just fell into place, how to modify noise makes so much more sense now
zzz :) thus demonstrating the benefits of weekly status...
zzz if you were psi, you'd be checking in a lot of f-bombs soon...
eyedeekay I'm relieved because I'm pretty sure life just got like, 60% easier for me at least, but I also think I have to delete a bunch of code
eyedeekay Glad I know now :)
zzz yeah, it's messy, because Noise is (and is documented as) a state machine, which is unlike any spec I've ever read before or since.
zzz our docs "unroll" that state machine into a more traditional protocol spec
eyedeekay Yeah it's different enough that I missed a lot of the relationship between the two specs until now
zzz thus making it possible to implement without ever looking at the Noise spec
zzz but it makes it a lot harder to map between the two flavors, which you need to do if you're using a lib
zzz since the go lib looks like it matches the java lib pretty closely, you should be able to get a lot of hints from the java code
zzz our java code to generate message 1 is OutboundNTCP2State.prepareOutbound(), all of 75 lines and could be half that
eyedeekay For me it's NewSessionRequest, I'll compare them