#saltr (irc2p) | I2P+ 2.7.0-11+ >> i2pplus.github.io | skank.i2p | git.skank.i2p | purokishi.i2p | vituperative.github.io/i2pchat | natter.i2p | notbob.i2p | ramble.i2p | shreddit.i2p | cake.i2p | /msg an op for voice

weko zzz: What is reference for streaming protocol and realization?

orignal basically what is it based on?

orignal such NACKs is veru exotic thing

weko I will do tests with UDP, but I guess most problems with speed in streaming protocol

weko especially with SSU2 as transport

weko streaming protocol or/and realization *

zzz weko, orignal, streaming is all jrandom 2003, before my time

weko Realization also?

zzz what do you mean by realization? it's originally jrandom, but we've done a lot of fixes in 20 years

tennis2 Eye-eye peers, its me again. Re the above re above "i2psnark dropping all the metadata" (and re-reading disks) has anyone looked at the code, does it check out?

tennis2 cake.i2p/view/ezPiwVHCp1_ZUXPNvn3TkHvRDjRn7oUyvjiTbEF7Z_UJhsu28pum/ezPiwVHCp1.txt

tennis2 If you like it and like the idea of not re-reading all the torrents on the disk it would be great if you might add it to your repo, with the above sudo command to autostart it.

orignal zzz but maybe you know what was the underlying protocol

orignal the questuion is why special protocol was required rather than ordinary TCP?

zzz orignal, weko i2p-projekt.i2p/en/docs/api/streaming see history section at bottom

zzz the full streaming implementation was deployed with a sliding window protocol and optimizations to take into account the high bandwidth x delay product.

zzz some reasons:

zzz dest vs. IP; 0-RTT delivery; signatures

zzz you would have to write a user-mode TCP impl even if you did TCP

zzz you can find a lot of the 2004-2005 discussion in the gmane.network.i2p newsgroup at NNTP news.gmane.io

zzz you can either load it up in thunderbird or I might have a tarball somewhere

zzz found it

zzz zzz.i2p/archive/index.html

zzz ^^^ orignal weko

orignal yes I'm reading

orignal I know I have to implement own TCP but it ould be a strdard thing

orignal in our opinion NACKs cause a lot of inefficiency

weko I don't know about nacks

weko But I guess most problem is streaming. As I said, I will test with UDP

orignal every time we receive a NACK it produced severe speed degradation

zzz orignal, weko, I have some advice if you're working on streaming

zzz don't do it on testnet (at least not at first) - it's too messy, too hard to get reproducible results

weko [15:56:53] <zzz> don't do it on testnet (at least not at first) - it's too messy, too hard to get reproducible results

zzz you want to do standalone loopback tests

weko Ye I know

zzz i2ptunnel client to server

zzz and use wget or curl on a big file

weko But some bugs we can see even at testnet

zzz but you have to put delay and drop in the middle, you have to write that code if you don't have it

zzz test say 100ms to 45s delay, and 0 to 10% drop

weko zzz: I used wget, but wrote special tool

zzz keep careful records, make charts or graphs of bandwidth vs. delay and bandwidth vs. drop

zzz change only one thing at a time

zzz look for how fast it recovers from stalls or if it gets "stuck" at a low bandwidth

zzz also track retransmission % and window size trends

weko I guess best way to repeat TCP algos

weko But TCP doesn't have NACKs

zzz right. the RFC I gave you the other day is important

weko And for i2p we can do multipath

zzz acks and nacks is the same thing, don't let it confuse you. nacks just tells you where the "holes" are

weko I need more research

zzz yeah we spent years on it. Don't make quick changes; test test test before checking anything in

zzz if you can go i2ptunnel-streaming-i2cp you could use my i2cp loopback tester so you don't have to write anything... but I don't think you support streaming to i2cp?

weko Why window size maximum is 128?

weko I guess you have better implemention than i2pd, then why so low window?

zzz don't know, but that's really big

weko Nope

zzz what's our max bytes? 1815? that's 232KB in flight

weko Just calc speed for typical i2p 3+3 ping - 500ms, and with mtu

zzz sure but you have to send 232KB w/o any loss to hit 128 window

zzz what window sizes are you seeing on testnet?

weko Maximum 400KB/s with 500ms ping

zzz with no drop?

zzz but what's the window size

weko [16:10:54] <zzz> with no drop?

weko Ye i2pd can

zzz and RTT and RTO

weko zzz: I set 4096 maximum, and saw it

zzz I doubt it, that's 7.5 MB in flight

zzz you shouldn't be buffering that much in the routers anyway

zzz add 0.5 or 0.25% drop to get a more realistic test

weko Okay for 500 ping and 14MB/s

weko zzz: I added

weko 1%

weko Can test more

zzz 500 round trip or per hop?

weko So it works only with ntcp2 because bad steaming realization

weko zzz: round trip

zzz ok

weko So unrealistic on real net

weko But 128 really to small

weko Too

zzz disagree. 128 is fine in real world.

zzz you can't send 7.5 MB at once, nobody has a buffer that big, it will all get dropped along the way

weko zzz: when routers can give to us 1MB/s, we have only 400KB/s

weko So I just suggest increase to 256

zzz there's a lot of overhead

weko But I know I need note tests etc

zzz if you're going to 4096, you probably have bugs. It shouldn't go that high

zzz are you doing AIMD?

weko zzz: I agree about 4096

zzz you need proper AIMD algorithms

zzz the goal is not only to go fast. you also have to be _efficient_. you can't retransmit or nack like crazy

weko What is AIMD

weko All In Mind?

zzz en.wikipedia.org/wiki/AIMD

weko We don't use AIMD

zzz when you get a nack, cut your window in half

zzz actually, now we cut it to one I think

zzz you have a LOT to fix then

weko I know)

zzz this is basic stuff

weko Agree

weko Anyway window 128c can sometimes limiting i2p speed rn

weko 128*

zzz ok, but once you do AIMD it won't be a problem

weko With bad rtt but good bandwidth

zzz actually we are now using TCP Westwood+ algorithms everywhere

weko Hmm

zzz we spent about a year converting everything to Westwood+

zzz and fixing our timers

zzz here's the references from our javadocs"

zzz **

zzz * A Westwood+ bandwidth estimator with

zzz * a first stage anti-aliasing low pass filter based on RTT,

zzz * and the time-varying Westwood filter based on inter-arrival time.

zzz *

zzz * Ref: TCP Westwood: End-to-End Congestion Control for Wired/Wireless Networks

zzz * Casetti et al

zzz * (Westwood)

zzz *

zzz * Ref: End-to-End Bandwidth Estimation for Congestion Control in Packet Networks

zzz * Grieco and Mascolo

zzz * (Westwood+)

zzz *

zzz * Adapted from: Linux kernel tcp_westwood.c (GPLv2)

zzz *

zzz * @since 0.9.46

zzz */

weko Oh then we know reference

weko It's good

zzz if you are hitting 4096 window with 500 ms ping and 1% drop, you are very very very broken. You should be more like 20 window size for that

weko zzz: yes I know. But I had 0% drops at tunnel layer

weko I understand about what you

zzz right, that's why you need to test standalone so NTCP2 or SSU2 don't "fix" the drops for you

zzz because the java routers in the network do a LOT of dropping, so you either need java routers in your testnet, or a standalone test

zzz a java router is not going to buffer 7.5 MB

weko zzz: i2pd also drops

weko Packet permutations is problem for i2pd rn

weko Happens in ssu2 with i2pd and I guess with ntcp2 and ssu2 in java

weko I really need sleep... with, in... Damn

weko zzz: transits not necessary to have such buffers

zzz you also have to be efficient. a TCP ack is 40 bytes. A streaming ack is 1028 bytes thru a tunnel. you can't be sending acks and nacks like crazy

zzz what router bandwidth limits are you using on the testnet?

zzz you may also wish to use i2psnark standalone in debug mode to see what our typical window/RTT/RTO/etc. values are