IRCaBot 2.1.0
GPLv3 © acetone, 2021-2022
#saltr
/2024/01/19
~dr|z3d
@RN
@RN_
@StormyCloud
@T3s|4
@T3s|4_
@eyedeekay
@orignal
@postman
@zzz
%Liorar
+FreefallHeavens
+Leopold
+Xeha
+acetone
+bak83
+cancername
+cumlord
+hk
+profetikla
+uop23ip
+weko
An0nm0n
Arch
Danny
DeltaOreo
Irc2PGuest21357
Irc2PGuest21881
Irc2PGuest43426
Meow
Nausicaa
Onn4l7h
Onn4|7h
Over1
anon2
anu3
boonst
mareki2pb
not_bob_afk
plap
poriori_
shiver_
simprelay
solidx66
thetia
tr
u5657
eyedeekay weko re: proposal status still undecided. I think it's a good idea but we'll need time to discuss it.
weko eyedeekay: okay I'll not write anything
dr|z3d I've been giving snark's single destination limitation some thought recently.
dr|z3d If snark didn't create tunnels and its own dest, and delegated that, then it's possible multiple dests could be nominated to handle traffic.
dr|z3d Different dests for inbound and outbound traffic, non-persistent destinations per torrent, dests that automatically cycle, that sort of thing.
weko I hope final (from me) version
weko zzz, please edit meta/content if needed and add to proposals.
weko sorry. final final version:
weko Markdown, if somebody needed (can be not the same):
dr|z3d have you got that in esperanto, weko?
weko dr|z3d: idk esperanto... but want.
weko If you about esperanto version readme in one of my git repos.... So it's translator as I remember.
weko I'm already tires of doing this proposal
weko tired
dr|z3d I was joking, weko. I have no interest in esperanto :)
dr|z3d esperanto's about as useful as a marzipan dildo, and much less edible.
weko dr|z3d: oh missunderstood joke
weko didn't understand*
dr|z3d it's ok, weko, obscure joke :)
weko Jokes direct understand typically for me
not_bob I second the idea of dynamic non-persistant i2psnark dests per torrent. It would be better for privacy.
not_bob orignal: I sent you a message a few days ago about having option in the tunnels.conf to allow idle tunnels to auto-close and re-open later if traffic is detected.
not_bob orignal: Also, clearnet outproxying does not seem to work if I make a second http tunnel in my tunnels.conf. Thoughts?
not_bob orignal: The goal is to get a new dest when the tunnel re-opens.
dr|z3d support for multiple outproxies would be handy too
orignal multiple outproxy? how?
orignal I mean how do we select one
dr|z3d you have a list, choose one at random until it fails, choose another thereafter.
dr|z3d or implement your own selection strategy.
dr|z3d you might designate one as primary, and the others as secondary, or primary, secondary, tertiary etc, and select in that order.
dr|z3d extra points if you actually test a clearnet address with the nominated proxy to check that it's working before deciding to use it.
dr|z3d I was trying to explain to zzz a while back the usefulness of this particular strategy. not good enough to just check that the b32 responds.
orignal so round-robin?
orignal another option is balance
dr|z3d sure round-robin, balance, random.
dr|z3d if you're opting for balance, then you're talking about using more than one at a time.
dr|z3d which isn't necessarily a bad idea, but you'd want an option to cluster certain requests/hosts to a single outproxy in some instances, otherwise you'll get cross-site issues and wotnot.
dr|z3d zzz: those new sidebar notifications for the news feed, are they specific to router updates, or do they trigger for all news feed updates (ie blocklists) ?
zzz not sure, suggest you look at the diff
dr|z3d needs a bit of a buff up, either way.
zzz whats the buff?
dr|z3d putting something useful up there other than a b32.
dr|z3d anything other than the b32 would be better.
zzz we could do a reverse lookup, MRs welcome
dr|z3d not entirely sure by looking at the diff if non-news updates are causing notifications.
dr|z3d if (!langChanged()) {
dr|z3d long lastMod = NewsHelper.lastUpdated(ctx);
dr|z3d if (lastMod > 0)
dr|z3d _lastModified = RFC822Date.to822Date(lastMod);
dr|z3d a reverse lookup to get the hostname you mean?
zzz yes
dr|z3d is there even a hostname for the news feed?
zzz not sure
zzz the langchanged stuff is just to reset the last-mod after you change languages so you get the new one, which I snuck into the same checkin iirc
dr|z3d if there is, it should be specified in the configs as the feed, not the b32, and added to the default hosts list. if there isn't, then it should be registered and then used. but I was thinking more about displaying something useful there to notify the user.
dr|z3d if the blocklists get updated via the feed, then what about telling the user they've been updated. "20 new entries blocklist entries via foo.i2p" or whatever.
dr|z3d if it's a news item, "new news item by idk downloaded" or whatever.
dr|z3d right, my bad.
dr|z3d if (_failMsg != null) {
dr|z3d // from checkForUpdates()
dr|z3d _mgr.notifyComplete(this, "<b>" + _failMsg + "</b>");
dr|z3d } else if (_showStatus) {
dr|z3d if (status == 200)
dr|z3d _mgr.notifyComplete(this, "News updated from " + _currentURI.getHost());
dr|z3d _mgr.notifyComplete(this, "No new news available from " + _currentURI.getHost());
zzz happy buffing, MRs welcome ))
dr|z3d I'm not convinced of the utility of it right now. Maybe I'll get it.
zzz the two news servers are hardcoded b32s, not hostnames, to ensure they work no matter what addressbook shenanigans or downstream packaging things happen
dr|z3d well here's an idea. why not hardcode them with a reserved hostname, there are a few available.
dr|z3d news.router.i2p or whatever.
zzz I thought I did what you guys were asking for, but apparently I missed the mark? Still not sure quite what the complaint is
dr|z3d this is different. the unsigned update notification is great.
dr|z3d I thought you were going to replicate that for signed dev updates, but you went off piste :)
zzz I thought I stayed right on the piste
zzz I adapted the unsigned stuff over to the news fetcher
dr|z3d yeah, I see that. but we already have a sidebar notification system, persistent, that indicates the currently available news. if the transient notification indicated *what* was updated (news/blocklists) with a bit more info like the published date, author or number of entries, then it might get a pass. :)
dr|z3d and we also have persistent info on /configupdate that indicates when the news was last updated.. so just showing that the news was updated in the sidebar is redundant. that's my point.
zzz my understanding of the request was 'show more info in sidebar on results after clicking manual check-for-update, rather than just 'no new update available'
zzz in particular, failure info
zzz did not hear a request for 'tell me exactly what components updated on success'
dr|z3d yeah, that's broadly correct, but relating specifically to router updates, unsigned and otherwise, at least in terms of mesh's original moan/request.
zzz so maybe that's the difference between my piste and yours?
dr|z3d could be :)
zzz it's also possible that you're disagreeing with whomever made the request (mesh?)
dr|z3d no, I think it was a valid request to inform the user when connecting to the router update server failed, that was the original request in essence.
dr|z3d because before that it indicated "no update available" which was potentially misleading.
weko Clearnet access should be done with different software, as I think. And all features (multiple proxies etc) should be in that software.
weko Or like plugin...
dr|z3d weko, the problem with i2pd is that you can only specify a single outproxy.
weko dr|z3d: i2pd should not support outproxy ideally.
weko It's can be done with other software
weko zzz: did you add proposal?
zzz yes see link above
weko Nice
weko Thanks
dr|z3d weko: you can't use i2p's outproxies with other software.
zzz np. if you have updates please submit as a diff vs. the file in github, which I did have to tweak some
weko dr|z3d: I can and I use
dr|z3d no, not i2p's proxies. you need i2p to access them.
dr|z3d and ideally i2p would have a huge number to choose from, not none.
weko acetone outproxy exists for minimum
weko Not really "none"
dr|z3d well if that's available via clearnet, great.
weko I done tunnel for outproxy and connecting to clearnet IRC servers with setup this tunnel address port as proxy in irc client
weko Don't need really "outproxy" feature in i2p router
weko really need*
dr|z3d but that misses the point. and I'm not saying we don't have any, I'm countering your argument that "ideally i2p should not support outproxy".
weko dr|z3d: I was wrong. Meant "i2p router"
weko Not i2p at all
weko Because i2p support any connections. And connections for clearnet proxy also supported anyway
dr|z3d if I patch, I'll let you know, zzz, but my inclination right now is to remove the news related notification.
dr|z3d the more you say, weko, the more confused I am.
dr|z3d java i2p at least supports multiple outproxies, and on a single tunnel.
zzz would be nice to get feedback from mesh on it; I don't know how to reconcile your piste with his really
dr|z3d i2pd apparently only supports 1.
weko dr|z3d: ye I said ideally i2pd should not support outproxy at all
dr|z3d yeah, that boat has sailed, zzz, I prompted him months ago to offer something by way of feedback for your work, and he didn't.
weko Because unix philosophy
dr|z3d more "chatty" notifications for (un)signed dev router updates, great. perfect. that's what he asked for, that's what you delivered. perfect.
zzz most of the changes only kick in if you click check-f0r-update manually which I doubt people do often, so imho not worth a lot of drama either way
dr|z3d sure, weko, sure, you can create a bunch of tunnels to different outproxies and knit them all together with a backend, but that's not something every user wants to do. some users just want a one size fits all single port they can proxy all their traffic through, .i2p or clearnet.
dr|z3d true, zzz. minor.
weko dr|z3d: can be done with one proxy at userside, one destination in i2pd with software
weko And with multiple outproxies*
dr|z3d I only mentioned it because, having not seen it until earlier today, it was brought to my attention by someone who had no idea what it was. but yeah, minor.
dr|z3d you're adding a huge amount of complexity for something that shouldn't require it, weko.
weko dr|z3d: you do same with java i2p
dr|z3d no, in java it's simple. you nominate 1 or more outproxies, and that's it. java randomly chooses one.
dr|z3d java will check the dest is up or cycle to another.
dr|z3d if java checked a clearnet endpoint to validate the proxy was up, would be even better. but still, it's streets ahead of i2pd.
zzz the generic argument that 'java i2p has x feature, i2pd should do it too' rarely if ever gets any traction, ditto vice versa
zzz it's just not compelling on its own, and it also (if regularly adopted) stomps out product differentiation
dr|z3d there's nothing generic going on here, it's more a specific deficit that i2pd has that's been noted. whether orignal thinks it's worth considering is another matter.
dr|z3d as for java, with multiple outproxy support, we can fill in the gaps with software without the same level of complexity required to get i2pd working with multiple outproxies.
dr|z3d but there's always room for improvement, regardless of the implementation. a high tide lifts all boats..
zzz sure
dr|z3d have you thought any more about rate limiting strategy for keep-alive?
zzz point is that requests from users, rather than from fellow devs pointing out feature deltas, rightly get more attention
zzz no good ideas yet
dr|z3d notbob made the request, I just puffed it up :)
zzz once a feature is put on the list, that's the time to eyeball the other guy to see how he did it
dr|z3d so, rate limiting. max connections per client is one way to keep things sane.
dr|z3d concurrency, in other words.
dr|z3d buffers could be another way to implement some sort of throttling.
zzz that doesn't do anything if you can make 10k requests per minute on one connection
dr|z3d you know, instead of max requests per minute, clients are allocated a buffer instead.
zzz what are we buffering?
dr|z3d the amount of transferred data. buffers, or buckets..
dr|z3d and/or the number of requests made.
dr|z3d not so much requests, hard to peer into the keep-alive connection to identify those. but data transferred in a given period might be something to consider, with a queueing/burst strategy.
zzz don't start with implementation. start with what do we limit and where in the protocol stack. and/or how/where do we repair the current limiter that will be bypassed via persistence
dr|z3d throttling would be at the application layer, no?
dr|z3d ie where we do it now, more or less.
dr|z3d and repair is what I'm suggesting, instead of throttling by # of requests, we throttle by data transfer instead.
dr|z3d so instead of 50 requests/m, we throttle at 1MB/m or whatever seems suitable given the server context.
dr|z3d the problem with either implementation is that they can be trivially bypassed with a bit of effort.
zzz the layer choice is streaming vs i2ptunnel
zzz the counters are in streaming now
zzz but only i2ptunnel knows about add'l requests on same conn
dr|z3d probably a combination of both. i2ptunnel retains its ability to throttle based on requests, streaming handles data transfer?
zzz so the options are 1) add counters to i2ptunnel; 2) use accessfilter in i2ptunnel; 3) add something to socketmgr API that i2ptunnel can call, e.g. gotANotherRequest() to increment the counter and accept/rej
zzz any layer can and does reject
zzz but i2ptunnel can't get to streaming's counters atm
dr|z3d ok. so we keep the connection throttler broadly the same.
dr|z3d it serves a purpose, will carry on serving a basic purpose.
dr|z3d the filter gets beefed up to handle data rates.
dr|z3d does that sound vaguely coherent and viable?
zzz not really ((
zzz now we're counting "data rates" instead of requests?
dr|z3d data transfered per defined window.
dr|z3d which of course in the filter can be much more fine-grained.
zzz 'count something different instead of requests' is orthogonal to 'fix counting requests'
dr|z3d that's not the premise. the premise is "how do we throttle clients if not by requests?"
dr|z3d counting requests within a single keep-alive connection, forget it.
dr|z3d unless you want to implement some sort of java wireshark feature. which you don't. :)
zzz so far, I'm thinking of 'fix counting requests', and I laid out 3 options above to do that
zzz per-far-end counters of anything are relatively expensive, bw estimators even more so
dr|z3d not easy. the problem is that with keep-alive, counting requests is mostly a meaningless metric.
dr|z3d (as you pointed out yourself)
dr|z3d so buckets or buffers would be one way to mitigate that. once you've spent your allocated buffer, you're in a holding pattern until the next window.
dr|z3d or you use the allocated bucket and window size to determine the data rate.
zzz we already have bw estimation in streaming, but there's no configurable throttle there, and no aggregation of all conns to same dest
dr|z3d maybe lose the connection throttler altogether and provide a ui an i2ptunnel hookup for the tunnel filter is the way to go, seeing as there's plenty of overlap there.
zzz sounds like a lot of work
dr|z3d you mentioned possible url-based filtering using the access filter, so you've already sort of considered it. but yeah, it's not a trivial fix.
zzz anyway, all theoretical at this point, need two MRs approved before we get there
dr|z3d how far are we away from the server end of the keep-alive deal?
zzz client-end merge + a few days
dr|z3d roger that.
dr|z3d how's it all looking from your seat?
zzz + server end review/merge
zzz maybe still a little flaky?
dr|z3d still git.idk issues or those resolved now?
zzz seems ok