not_bob
zzz: Thank you.
dr|z3d
hit orignal up with your multi outproxy request, orignal, he wants to hear your thoughts :)
dr|z3d
*not_bob
not_bob
Does he?
not_bob
It might be better to wait till I have it typed up nicely, rather than just some haphazard comments here.
not_bob
I'll get on that later today and throw it his way.
dr|z3d
<orignal> I don't see him for awhile
dr|z3d
<orignal> yes close on idle is old story
dr|z3d
<orignal> I need him to discuss outproxy issue
not_bob
orignal: Close on idle would be a very useful option.
not_bob
orignal: It gives much better privacy as the dest for you browsing isn't the same your entire session.
not_bob
orignal: It's better to not have the same dest all the time as it allows someone to correlate your browsing.
dr|z3d
I think he was more interested in the request for multiple outproxy support, not_bob, could be wrong.
not_bob
Sorry, doing many things.
not_bob
Anyway, support for more than one http proxy is very useful. But, each of them needs to be able to outproxy as well. As it stands the only http proxy that can proxy is the one defined in i2pd.conf.
not_bob
I can configure more http proxies using tunnels.conf, and they work fine. But they can never outproxy outside of the I2P network.
not_bob
The basic idea is that you setup say 5 http proxy tunnels in your tunnels.conf, each with an idle timeout, or a way to reset just that one http tunnel's dest and connection (reset) every so often.
not_bob
And then you use another program to fetch resources from a random tunnel for each request.
not_bob
As the tunnels cycle on and off the dests change.
not_bob
Which gives you a series of ever changing dests for your browsing.
not_bob
Which in turn gives you a major increase in privacy.
not_bob
But, the key is we need mutli outproxy support for this to be a viable thing, and the tunnels need to be either reset manually (just one, not all) for a new dest, or the ablilty to go idle and then come back with a new dest when activity is detected.
snex
could make it so that the only thing the user cares about is the load balancing thing which stays on port 4444 and the others are all internal
not_bob
And, even if this use case is not common, the desire for more than one dest for browsing is likely to be a very good idea for people to segment the things they do.
not_bob
And the ability to idle timeout and close unused tunnels (if enabled to do so in the config) is a major boon to privacy as well. As you would not have the same browisng dest for days or longer at a time.
not_bob
snex: That's one way to do it. I've been using another port and leaving the factory http proxy tunnel alone.
snex
is there any reason to not just create a new dest on every http request?
not_bob
Yes, bad idea.
not_bob
It takes time to build a new tunnel. Yes you can do that, but it would be super slow.
not_bob
The idea is that you have 2x the tunenls you need running. And you switch which ones you are using every so often. This allows the other half to idle out and get a new dest.
not_bob
Then they are ready when you go to use them again later.
not_bob
zzz: I bumbed that.
not_bob
bumped....
dr|z3d
LOL
dr|z3d
don't be going around bumbing things, not_bob :P
not_bob
My typo? :)
not_bob
I do that all the time. Just ask my wife :(
dr|z3d
you'll get the wrong kind of reputation :)
not_bob
lol
not_bob
Yeah, that is best avoided.
dr|z3d
new dest on every request, snex?
dr|z3d
that's insane.
not_bob
Possible, but very bad idea.
dr|z3d
the cost to build tunnels would be enormous, and the latency would be ridiculous.
dr|z3d
otherwise, great idea.
not_bob
A 300 baud modem would be faster.
not_bob
When I was a kid I had a 300 baud applecat modem. I loved it.
dr|z3d
what you want is a rotating set of destinations that are renewed frequently enough to prevent correlation but not so frequently that you introduce any delay in requests.
dr|z3d
as a side benefit, you're much less likely to encounter site throttling.
not_bob
Yes
dr|z3d
bake in some connectivity tests to ensure your tunnels are performant, and you have something useful.
not_bob
I think it's a fun idea.
orignal
not_bob so you are saying that outproxy doesn't work in httprpxy tunnel?
orignal
zzz, a question for you. Say I receive TBR and recognize that next peer is too slow. What code should I put?
weko
Or transport to next peer is overloaded
not_bob
orignal: Correct. If I create a second tunnel in tunnels.conf it works fine for I2P sites, but not for clearnet sites via the outproxy. I've tried passing on the flag outproxy =
not_bob
It works fine in the tunnel defined in i2pd.conf, but not if I make a second one.
not_bob
Unles I'm doing something wrong, but I doubt that I am since the tunnel works otherwise.
not_bob
Anyway, I'm testing a few things to see if they are possible.
orignal
i will check
not_bob
Thank you
dr|z3d
the other issue is not being able to associate more than one outproxy with a single http client tunnel.
weko
dr|z3d: you said already
dr|z3d
just restating so we've got the full informal proposal(s) in one place, weko.
orignal
not_bob it works fine
orignal
checked right now
orignal
[Proxy-144]
orignal
type=httpproxy
orignal
port=4450
orignal
keys=transient
orignal
outproxy=http://<b32 address>
not_bob
Let me test it again.
not_bob
Do I need the there?
orignal
no diffience from standard http proxy
not_bob
I don't have that in my i2pd.conf file for the outproxy.
orignal
then it will not work either
orignal
you set either http or socks
orignal
but socks has diffrent meaning
orignal
it refers to local proxy
orignal
not remote
orignal
only stand for remote proxy
orignal
for remote socks you don't need a special tunnel just regular client
not_bob
I'm talking about outproxy=http://b32
not_bob
I get this "Host example.com is not inside I2P network, but outproxy is not enabled"
not_bob
With, or without the outproxy=http://b32
not_bob
And with or without outproxy=b32
orignal
works fine for me
not_bob
I don't want socks.
orignal
I don't know which outproxy do you use
orignal
I use one of mine
not_bob
It's a http proxy tunnel, and it works fine for inside I2P.
not_bob
type = httpproxy
not_bob
Hmm
not_bob
So, it should work then?
orignal
I told I tried it 5 minutes ago
not_bob
stormycloud is what I am using right now.
orignal
give me the address and I will try with them
not_bob
exit.stormycloud.i2p
weko
not_bob: do you need access only for clearnet?
weko
to*
StormyCloud
stormycloud is the best cloud
not_bob
I want access for I2P and clearnet.
not_bob
Here is my tunnel config for that tunnel.
weko
<StormyCloud> stormycloud is the best cloud
weko
:)
dr|z3d
the best cloud is no cloud at all :)
not_bob
I like to make my own cloud :)
not_bob
Much safer than OneDrive :)
StormyCloud
anything safer than OneDrive
not_bob
;)
orignal
everthing works fine through exit.stormycloud.i2p
StormyCloud
:)
not_bob
StormyCloud: Good to see you!
not_bob
orignal: What's wrong with the config I posted then?
orignal
idk
orignal
[Proxy-144]
orignal
type=httpproxy
orignal
port=4450
orignal
keys=transient
not_bob
Heh
orignal
outproxy=http://exit.stormycloud.i2p
StormyCloud
@orignal, I dont log anything so cant see any traffic.
orignal
that's all I have
orignal
you can't see traffic, just requests
not_bob
I'll try it like that.
dr|z3d
yeah, he's not paying any attention to those. if you connected over his outproxy, we believe you :)
not_bob
Nope, still get "Host slashdot.org is not inside I2P network, but outproxy is not enabled"
not_bob
And I'm doing a reload of tunnel config to reset the tunnels.
orignal
don't relaod
orignal
restart completely
not_bob
Ok
orignal
I'm not sure that reload wotks for outproxy param
dr|z3d
iplocation.net/find-ip-address to verify outproxy address.
orignal
but this is differen issue
not_bob
Ahh
not_bob
That was my problem!
not_bob
So, it does work, but it requires a full restart to work.
orignal
yes it needs to be fixed
not_bob
Ok, that would explian why I could not get it to work. Thank you.
not_bob
And it does work now, tested.
orignal
I will check the issue with relioad when I have time )))
not_bob
Thank you orignal.
dr|z3d
one issue down, one to go. well done, orignal
not_bob
orignal: Don't keys default to transient if one is not speificied?
not_bob
orignal: At this point, I'd like to associate more than one outproxy to a tunnel. The order does not matter, but if one is down, then it should use the next.
dr|z3d
acetone: around and about?
dr|z3d
if you are, can you pm StormyCloud your ipv6 address for outproxy.acetone
orignal
yes transient is default for cleint tunnels
orignal
no I don't have an ability to specify multiple outproxies yet
orignal
no acetone is not around
orignal
* [acetone_] idle 153:51:27, signon: Sun Jan 14 12:02:45
dr|z3d
don't blame him, fresh air and all :)
dr|z3d
are you contemplating multiproxy support, orignal?
dr|z3d
"yet" sounds like you may be.
orignal
not now
orignal
I have many other things to do
orignal
and I don't have time
dr|z3d
ok, I gotcha. not a priority, but maybe in future, then.
zzz
orignal, code 0 (accept)
orignal
but we don't want to accept
orignal
we want to reject
orignal
because we know that that path is overloaded
orignal
but maybe use different than 30 code
orignal
because we are capable but that link is not
zzz
disagree. accept it.
zzz
then the 'overloaded' router can accept or reject, and the tunnel creator can discover for himself how fast it is
zzz
maybe it's an exploratory tunnel.
zzz
if you reject it, creator will blame you
orignal
that's why I would prefer to use different code
orignal
next router wouldn't discover it either
zzz
how do you determine 'path is overloaded'?
orignal
by outgoing queue for example
orignal
by actual bandiwtdh vs. published
zzz
does that happen often?
orignal
yes, we see it often nodays
orignal
we explude such links from own tunnels
orignal
also high RTT
orignal
dGk~: [2001:bc8:30c6:100::dead]:15133 => [306109186:374677537]
orignal
[queue:22]
orignal
=> UbsQ: 69.123.223.149:25277 [2405154:59714] [queue:4] [slow]
orignal
like this
zzz
there's no way to determine 'actual bw vs published'. you don't get all his bandwidth for one tunnel.
orignal
if I see actual 500 Mbs on a link while it publishes O
orignal
ofc idk his full banwidth but if see that my link exceeds his bandwidth I bypass
zzz
if I have a thousand connections, each one isn't getting the full bandwidth
orignal
it's not true anymore
zzz
I almost never see overloaded / backlogged connections. Maybe you have bugs?
orignal
there are reallty high loaded tunnels
orignal
maybe you don't print them?
zzz
you need to be throttling / dropping at the tunnel layer, don't let a tunnel overload a connection
zzz
that's how you fix it
orignal
please exaplain
orignal
if there is a high loaded tunnel
orignal
what do I do with data gooes trough?
orignal
limit per tunnel?
zzz
at some queue in the "middle" of your router, do your dropping there
zzz
not at the transport
orignal
yes, when it's too big
orignal
like 500 I2NP messages backlog
orignal
on a transport
orignal
and no there is no internal queue inside
zzz
yeah, that's too late.
orignal
only on links
zzz
you have to limit partitipating tunnel traffic before it gets to the transport
orignal
Tunnels:
orignal
Queue size: 25
orignal
it's nothing
orignal
what should I limit with?
orignal
what value do you use?
zzz
codel
zzz
we limit to the configured share % times the configured outbound bandwidth limit
orignal
but it's for all transit
orignal
not per each tunnel
zzz
no separate thread
zzz
they are waiting for bandwidth allocation
orignal
let me check
zzz
each tunnel has a RED queue
orignal
red?
zzz
random early drop (codel)
zzz
so they are throttled by the participating bandwidth limit
orignal
how do you distribute bandwidth between few transit tunnels?
zzz
the bandwidth limiter on the right in the picture is the total bandwidth limit
orignal
do you asiign some weight or all equal?
zzz
first come first served, nothing fancy. equal weight
zzz
but a bandwidth hog will overflow his queue
orignal
what if few tunnles have data to send?
orignal
who will be serverd first?
zzz
huh?
orignal
5 transit tunnles have messages to send thorugh a libk
zzz
we don't do round-robin
orignal
link became ready to send more
orignal
who will be serverd first?
orignal
then why do you need a queue per tunnle rather than per transport?
orignal
if it's always FIFO
orignal
a high loaded tunnl can still shit out whole transport
zzz
it's FIFO, but we use priority queues, so each message has a priority
orignal
how do you asiign priority
orignal
that's my main question
zzz
message size, our messages vs. part. messages, i2np message type, ...
zzz
the priority queues are marked in the picture
orignal
99% of ipnp message type is "tunnel" or "tunnelateway"
zzz
sure but all of our messages are higher priority than tunnel messages
orignal
good point
orignal
will implemnt this way
zzz
here's ours:
zzz
/**
zzz
* Priorities, higher is higher priority.
zzz
* @since 0.9.3
zzz
*/
zzz
public static final int PRIORITY_HIGHEST = 1000;
zzz
public static final int PRIORITY_MY_BUILD_REQUEST = 500;
zzz
public static final int PRIORITY_MY_NETDB_LOOKUP = 500;
zzz
public static final int PRIORITY_MY_NETDB_STORE = 460;
zzz
public static final int PRIORITY_EXPLORATORY = 455;
zzz
/** may be adjusted +/- 25 for outbound traffic */
zzz
public static final int PRIORITY_MY_DATA = 425;
zzz
public static final int PRIORITY_HIS_BUILD_REQUEST = 300;
zzz
public static final int PRIORITY_BUILD_REPLY = 300;
zzz
public static final int PRIORITY_NETDB_REPLY = 300;
zzz
public static final int PRIORITY_HIS_NETDB_STORE = 200;
zzz
public static final int PRIORITY_NETDB_FLOOD = 200;
zzz
public static final int PRIORITY_PARTICIPATING = 200;
zzz
public static final int PRIORITY_MY_NETDB_STORE_LOW = 150;
zzz
public static final int PRIORITY_NETDB_EXPLORE = 100;
zzz
public static final int PRIORITY_NETDB_HARVEST = 100;
zzz
public static final int PRIORITY_LOWEST = 100;
zzz
we also have application priorities, so snark is lower than http, etc
orignal
and when do you drop? when you can;t send right a way?
zzz
codel or queue overflow
zzz
let me look to see how big the tunnel queues are
orignal
because we need to do somthing with it
orignal
sometimes we send NTCP2 frame of literally 64Л
orignal
*64K
zzz
but main point is you have to drop in the "middle of the router" before you get to the transports.
orignal
e.g. max
zzz
don't let participating traffic kill you
orignal
yes, priority is good idea
zzz
we also drop more for obep and less for ibgw, because the message is "sooner" at the obep
weko
[13:22:45] <zzz> there's no way to determine 'actual bw vs published'. you don't get all his bandwidth for one tunnel.
weko
No way? It is possible and can realized.
weko
[13:32:10] <zzz> yeah, that's too late.
weko
Agree but need even increase maybe because i2pd steaming. Can fix streaming and then reduce queue.
zzz
still researching our tunnel queues, stand by...
weko
zzz: ideally do same warning to tunnel owner that transport overloaded
weko
For except data loss. Data loss very bad for streaming (creating big lags because big latency)
weko
For some tasks avoiding lags prefer then speed
zzz
mid-tunnel injection is not possible
weko
And yea, transport can be easy overloaded, because big packet loss ratio or low speed of transmit data
weko
zzz: can be added to protocol
zzz
no
weko
It's just bidirectional tunnels
weko
We use for data only one direction anyway
zzz
ok orignal after some research:
weko
We need more stable connections for really be "IP2".
zzz
we don't have per-tunnel queues except for IBGW
zzz
the spots marked "RED" in the middle are just a single "synthetic" queue limiting the participating bandwidth
zzz
as I was explaining to eyedeekay the other day about synthetic queues
zzz
IBGW has a red/priority queue, because it's important to do the dropping before you fragment, not after
zzz
so you know the message type and size for the priority calculation, and drop whole messages, not fragments
zzz
so the fragmenter pulls off of that queue
weko
Anyway with current i2pd steaming realization transport queue should be more then MAX_WINDOW * 2
zzz
opposite for OBEP - reassemble and then RED
weko
With better realization can be reduced
zzz
weko, this has nothing to do with streaming layer. this is low-level routing queue management / throttling
weko
I understand. It's bad steaming realization
weko
And it's reason for bad fixes at low level
zzz
maybe, maybe not.
zzz
router has to protect itself
zzz
see picture linked above
weko
I just realize how i2pd's streaming works. And how should for not be stucked by queue
weko
It's trying to send all packets after ACK at one time.
zzz
I don't have a picture for streaming congestion control ((
weko
So i2pd sends ACK every RTT/10
weko
So not important ))
weko
When we try send full window (when packet loss), we trying to send more then queue can accommodate
weko
Not accommodate*
zzz
you still need window, slow start, AIMD, congestion window, etc
zzz
weko, RFC 6298 is the best reference for what you should be doing in streaming
zzz
especially section 5
orignal
got it
dr|z3d
we've got a new webmail webapp someone's been keeping under wraps.
dr|z3d
sushimail.
zzz
derp. RU blocking Wireguard
zzz
no shit
zzz
that's why it took a year to do SSU2. We had to fix Wireguard
StormyCloud
bunch of optimization done on checki2p.com thanks to Mr. ~Dr
itsjustme
*** looks around ***
itsjustme
Hey dr|z3d!
dr|z3d
itsjustme!!! been a while! you good?
itsjustme
I've been well overall yeah
itsjustme
Been busy ish
dr|z3d
good, good
itsjustme
How have you been?
itsjustme
I was having issues staying connected for a while
itsjustme
Seems like it's stabilized a bit
dr|z3d
are you back with us, or just a flying visit? :)
itsjustme
Hopefully back but it has more to do with network stability
dr|z3d
ah, well, yeah, if you're up to date, things should be a little less volatile. not bad here, thanks.
itsjustme
Good to hear :)
itsjustme
I never left per se
itsjustme
It would just constantly disconnect so I couldn't really converse
itsjustme
How have things been in i2p land?
Liorar
Hey itsjustme - I remember your nick :)
itsjustme
Hey Liorar!