zzz
0) Hi
zzz
hi
zlatinb
hello
orignal
hi
zzz
what's on the agenda for today?
orignal
as usual SSU2
orignal
I have two items
orignal
1. please clarify how Acks are supposed to work
zzz
go ahead and list them, I have one also
orignal
2. Tranmission windows and RTT
orignal
I need to understand your algorithms to be on the same page
zzz
3) RI fragmentation in session confirmed
zzz
4) general status
zzz
let's do 4) first
zzz
I've been working on general cleanup and small bugs, nothing major
zzz
haven't finished peer test coding yet and haven't tested
orignal
so, SessionConrirmed works data pahase also works in both directions
zzz
been thinking about RI fragmentation which we'll get to in 3)
zzz
when do you plan to turn on publishing host/port so we can test java Alice to i2pd Bob? that's important to make sure nothing is broken the other direction
orignal
when I write the code for it ))
orignal
in couple days
zzz
that would be great
orignal
but I tested it before I stgarted testing with you
zzz
yes I know
orignal
so both side should work, but we will see
zzz
I'd like to lock down the basic protocol before the release in two months, I think we're well ahead of schedule for that
zzz
I'm restarting frequently as I fix things. Also finding some SSU 1 stuff and fixing that along the way
orignal
good
zzz
that's all the status I have, any other status from you?
R4SAS
hi
zzz
hi R4SAS
orignal
no, but you know my status already
orignal
I'm developing data pahse stuff
orignal
and need to implemend send and retrans
zzz
sure, but good to make sure everybody is on the same page
orignal
and not make the same crap as in SSU1
zzz
ok that's it for 4) then
orignal
yes
zzz
:)
zzz
1) ACKs
orignal
please explain
zzz
ask your questions, I'll do my best
orignal
I understand what is Ack trough
zzz
or I can start from the top if you want
orignal
but don't understand other part
zzz
you're asking about the format of the ACK block, or the general way acks work and what are they acking?
orignal
no. about a way
zzz
?
orignal
I don't understand how can I ack throusand messages for eaxmple
zzz
ok
zzz
here we go
zzz
wanted to make sure I understood the question because it's a big topic
orignal
my impression was that ack torugh is like in TCP/IP
orignal
e.g. ack of all message before that one
zzz
yes. ack through is usually the highest packet you're acking
orignal
but you specs says somthing ellse
orignal
it says it's ack for that message only
zzz
but it's not _all_ before, because we also have nacks, which are the gaps
zzz
an example:
zzz
ackthrough 10 acnt 2
orignal
I also understand what is nack
zzz
means we are acking 10 and 2 before that
orignal
similer to streaming
zzz
so we're acking 10, 9, 8
zzz
so ackthrough 1000 acnt 255 : that acks 256 messages
orignal
exactly
orignal
does it mean I can't ack 1000 in one ack?
zzz
that's where the ranges come in
zzz
example: ackthrough 1000 acnt 199 range 1 99 : that acks 1000-801, nacks 800, acks 799-701
zzz
or you could have ack through 1000 acnt 255 range 0 255 0 255 0 255 : that acks 1000 in about 10 bytes
orignal
let's start from simple thing
orignal
I want to ack 1000 messages and no gaps
orignal
I set acktorugh 1000 and one range 0-999 ?
zzz
ack through 1000 acnt 255 range 0 255 0 255 0 255
zzz
no because ranges are one byte of nacks and one byte of acks
zzz
so a range is 255 acks max
orignal
so, 4 ranges
zzz
the base ack through, acnt, and 3 ranges for the other 750 or so
orignal
what would acktrough 1000 and acnt 0 means?
zzz
this is all adapted from QUIC
orignal
just message 1000?
zzz
correct
orignal
that's what I wanted to know
orignal
it's neither TCP/IP, no streaming
zzz
yes, because it's not guaranteed delivery
zzz
so you only send each packet number once
orignal
now. how often do I include ack block?
orignal
for each packet?
zzz
if the other guy doesn't get it, it's a nack forever, until you shift up how much you want to ack
orignal
that's why I need to send more acks
zzz
right now, I'm putting an ack block in each packet that has room
zzz
but I'm fragmenting to max MTU, so there's no room with first fragment
zzz
which is different than SSU 1, where we left room in every packet
orignal
but how?
zzz
how what?
orignal
say I have received 1 million packets so far
orignal
what do I include into ack block?
orignal
since acktrough is not enough
zzz
right
zzz
so here's how I do it.
orignal
and I can't include ack for each packet
orignal
please tell me
zzz
I have a "bitfield" which tracks what I've received. You know, an array of longs
zzz
receive a packet, set the bit
zzz
and I have code that converts the bitfield to an ack block
zzz
with a max number of ranges, based on how much space I have
orignal
like in SSU1
zzz
but here's the deal. the bitfield doesn't get bigger and bigger.
zzz
It has an offset
zzz
so when it's almost full, I "shift" up, and "forget" the oldest part
orignal
so what's max number of packets do you ack in each packet?
orignal
255?
orignal
assume there is no gaps
zzz
right now I'm at 512, but that may be too big, 256 not a bad choice
orignal
and you don't send nacks
zzz
and shift up every 64 or 128
orignal
255 is fine
orignal
so you typycal ack block contains one range
orignal
got it
zzz
no ranges, just ackthru and acnt
zzz
which is kindof like a range, but I guess I'm calling "range" a NACK and a ACK field
orignal
I though acnt is nuumber of ranges
orignal
so it's number of message before acktrough?
zzz
no. acnt is the number of acks after ack through
zzz
before, right
zzz
number of ranges you get from the size of the block
zzz
two examples from my logs:
zzz
Got ACK block: ACK 3-0
zzz
that's ack through 3, acnt 3
orignal
after?
zzz
Got ACK block: ACK 200-173 NACK 172-170 ACK 169-0
orignal
then why it's 1 byte only?
orignal
make it 4
zzz
that's ack thru 200, acnt 27, range 3 170
orignal
and it will solve all the problems
zzz
for acnt?
orignal
yes
orignal
if I have stream witghout gaps
zzz
we don't need to ack a thousand packets. We just agreed we're only remembering 255
orignal
I will always have acnt = acktrought - 1
zzz
so if your bitfield is only 255, and there's no gaps, 1 byte of acnt is enough
orignal
probably
orignal
if not gaps
zzz
right
orignal
evething is clear now
orignal
thanks
zzz
github.com/i2p/i2p.i2p/blob/master/router/java/src/net/i2p/router/transport/udp/SSU2Bitfield.java if you want to take a look or steal anything
orignal
will do
zzz
it automatically "shifts up"
zzz
2) transmission windows and RTT
orignal
yes
orignal
I have ucoming queue of I2NP message
orignal
how do I send them?
zzz
so
orignal
in NTCP2 I make a frame send it and wait until socket is ready
orignal
in SSU2 I need to undertsand how I do it
zzz
transmission windows, congestion control, RTT, it's all really outside the SSU 2 spec (and the SSU 1 spec)
zzz
we can give some guidance, but we're not done either
orignal
I can't just dump all of them like I do in SSU1
zzz
you shouldn't do that in SSU 1 either :)
orignal
but what do you do now?
orignal
I doubt you send all of them
zzz
basically, zlatinb and I have been working for years on this
orignal
I know I shouldn't
zzz
both in SSU 1, and streaming, and elsewhere
orignal
that's why I want to see your results only
zzz
the basic guidance is to be "TCP-like"
orignal
so should I adopt streaming code for it?
zzz
and the design of SSU 2 makes that easier
zzz
it's hard to just copy paste, but you have to track RTT, and do fast retransmissions, and keep track of estimated bandwidth
zzz
so we hope with SSU 2, doing congestion control will be easier. and things can be simpler than in SSU 1
zzz
but basically everything is like TCP. RTT, RTO, congestion window, fast start, fast retransmit, ...
zzz
zlatinb, you have anything to add here?
orignal
let me formulate the quation differently
orignal
what is your inital windows and RTT?
orignal
sorry RTO
zlatinb
well, it's one of the reasons I was looking into nack-oriented protocols
zzz
initial RTO is I think 1 second
zzz
that comes from some RFC
zlatinb
don't know enough about SSU2 to tell how it's going to be different
orignal
how about windows size?
zzz
it's the same as in SSU 1, I think
zzz
stand by
orignal
I know but for me is nothing ))
orignal
I just flood the network
zzz
// RFC 5681 sec. 3.1
zzz
if (_mtu > 1095)
zzz
_sendWindowBytes = 3 * _mtu;
zzz
else
zzz
_sendWindowBytes = 4 * _mtu;
orignal
you mesure it in bytes rather than in packets?
zzz
yes
zzz
zlatinb, so far everything in SSU2 is the same as in SSU 1. Any differences, TBD, will probably be around NACKs and fast rtx
orignal
but I think it's worth to measure in packets in SSU2
zzz
actually in SSU 1 we have both packet and byte limits, but the primary congestion control is bytes
zzz
It's a complex topic, and almost all of it is "implementation detail" lol. We're happy to answer questions, but we've been working on it for years
orignal
I thibk only packets matter in UDP
orignal
so what sould be initial window size?
zzz
we've found that the TCP RFC's actually make the most sense to us, even for UDP
zzz
initial window size = 3 * MTU
zzz
see above code
orignal
so 3 packets?
zzz
I believe QUIC also does it based on bytes, not packets, but I'd have to reread it
zzz
no. 3 * MTU bytes
orignal
I would like to do it in packets
orignal
a full packet is 1 MTU
orignal
so 3 full packets
zzz
not going to argue, do what you like, but I suggest you read what QUIC does
zzz
correct, 3 full packets
orignal
I like to adopt streaming logic initially
zzz
ask questions any time
zzz
can we move on to 3) because it's important
orignal
yes
zzz
great
zzz
3) RI fragmentation in session confirmed
zzz
as you recall from last week, I said I would research and think about
orignal
please examplain what's an issue
orignal
yes
zzz
RI too big, won't fit in session confirmed, what do we do
zzz
option a) is what's in the spec now:
orignal
gzip or split
zzz
we have gzip now, still not small enough 100% of time
zzz
so spec says:
zzz
send first fragment in session confirmed
zzz
split()
zzz
get the rest of the fragments in the session and ack them in the session
zzz
until you have all the fragments and check the RI sig and s, the session is "unconfirmed" and you really shouldn't send or receive I2NP
zzz
that's doable, but a little bit of a mess
orignal
I have another idea
zzz
also, we'd need to send "i" in the session confirmed in a separate block, so bob can send acks
zzz
I have another idea also, but you go first
orignal
send identity only
orignal
and let Bob request us through netdb
zzz
right. we discussed that last week. we both agreed full RI is better
zzz
we could sign something, or, as you say, get through netdb
zzz
which wouldn't work for hidden mode
orignal
so what's your idea?
zzz
let's call yours b) and the next thing c)
zzz
c) jumbo message for session confirmed, send in multiple parts
zzz
so we indicate in header that session confirmed is fragmented
zzz
bob saves the bytes for each part
zzz
when he has them all, he puts them together, and THEN does the noise and chacha/poly
zzz
so you don't have a session at all until you have all the parts
zzz
it's one big encryption for the whole thing
orignal
besically few SessionConfirmed messages
zzz
yes, but only one chacha/poly payload, maybe 3000 or 5000 bytes
orignal
also uis had an idea
zzz
and in the header we put fragment 1 of 3 total or something
orignal
why do we have to be limited by MTU in this case
zzz
but it means there's no way to ack individual fragments for c)
zzz
alice would have to send them all
orignal
just send longer UDP packet and let network split it
zzz
may not always work, and linux likes to complain in the logs
orignal
e.g. can you send a 2K UDP packet?
orignal
kernel will split and assemble back
orignal
polistern did it for bote
zzz
I think in general, it doesn't work? not sure
orignal
by sending longer UDP datagrams
zzz
it's a good question but afaik not a great way to do it
orignal
polistern said it works
orignal
it's not a great way for regular traffic
zzz
sure, it _might_ work, but I don't think it would work every time, every OS, every nat/firewall
orignal
but for one message it might be a solution
zzz
ok that's d) then, let me write them all down
zzz
any thoughts on c) ?
orignal
c is basically "soft" version of d
orignal
you split your packet to fragments rather than kernel
orignal
it would work too
zzz
right. we split it up, and slap a header on each part after the first
zzz
the first header is mixhashed() into noise. the headers for all the other parts, just get the fragment number and throw the header away
zzz
if anything is corrupted you will find out when you glue it all together and give it to noise
orignal
yes
zzz
what I like about c) is it seems "safer" - you don't have this session that is "unconfirmed" for a while
zzz
also simpler in a way - you don't have multiple RI fragment blocks. It's one huge RI block
orignal
I can't go through woth it until I receive poly1305 hash
zzz
right
orignal
so I need eevrything
zzz
yeah
orignal
but yes
orignal
I don't need to send SessionCreated again
orignal
when I receive first fragment
zzz
the only problem with c) is I don't see a way to ack individual packets, because we don't have a session to send acks with
zzz
so you'd have to retransmit all the packets of session confirmed
zzz
but let's be serious, most of the time it's only two packets
zzz
I don't see any 3000 byte compressed RIs, at least not yet
orignal
you don't have to
orignal
Alice must keep resending all fragments
orignal
until she receives ack
zzz
agreed. no big deal, probably only two
zzz
so I propose we think about it another week, no decision today
zzz
that ok with you? talk about it in a week?
orignal
fine
orignal
yes
zzz
super
orignal
and we need to come to decision
orignal
because I'm going to publish RI with 7 addresses soon ))
zzz
yes, pretty soon
zzz
anything else for the meeting?
orignal
no
zzz
thanks everybody. knew this would be a long one
zzz
IPv6 does not support fragmentation/reassembly; I believe that rules out option d)