mesh
zzz: when I close a client socket the server gets a I2PSocketException, STATUS_CONNECTION_RESET
mesh
what does this status mean?
mesh
ls
mesh
oops
mesh
zzz: are you familiar with TcpNoDelay?
zzz
mesh, how may I help you?
mesh
zzz: hey
mesh
zzz: was curious about TcpNoDelay and I2P
mesh
what do you think about simulating Socket.setTcpNoDelay(true) by setting a very small WriteBuffer on a Socket
mesh
I2PSocketOptions.setMaxBufferSize(10) or even 1
mesh
good idea? terrible idea?
zzz
what's the problem you're trying to solve?
mesh
zzz: well from 10,000ft... I would like a way to telnet into a Java process over i2p without creating a bunch of server and client tunnels
zzz
so how does a small write buffer solve that problem?
mesh
telnet is a pretty simple protocol but it's really designed for "streaming" ... ie most telnet clients and servers invoke java.net.Socket.setTcpNoDelay(true)
mesh
I was thinking I could simulate this by setting a very small write buffer
zzz
i think that would perform horribly, if we even allow it
zzz
streaming + i2p has enormous overhead compared to TCP
zzz
we have a 'passive flush delay' param internally, but there's no config option to override it
mesh
zzz: I2PSocketOptions has a method setMaxBufferSize
zzz
you asked good or terrible idea, my answer is terrible
mesh
but you're saying that the 'passive flush delay' timer would prevent real time streaming
zzz
not saying that at all
zzz
but if you're concerned about latency, you're not going to be happy with i2p
zzz
if you want to implement telnet or anything else where you know when you've written all the data, then call flush()
mesh
zzz: ok. it's interesting because irc seems to mostly perform well over i2p and it is also a "streaming text protocol". Is this something the IRC tunnel optimizes?
zzz
does it call flush() ? probably
zzz
my point is there's nagle-like delays at multiple protocol layers, in every router your data travels through.
zzz
this is for efficiency given the large overhead that i2p adds at each layer
mesh
yeah it's a good point. obviously TcpNoDelay doesn't make any sense for i2p. But in the general case of streaming data as fast as possible it sounds like the best bet is just to reduce the hop count and call flush often
zzz
flush() will save you 100 ms or so, but don't call it unless you know you're out of data
zzz
my irc client is reporting 0.5 sec round trip, that's not bad through 13 routers. flush() will get the data started out the door of your router but obviously won't do anything about the delays in the other 13
mesh
zzz: yeah I think for something like irc it's not so bad. irc is a line-oriented protocol and what I envision is roughjly the same
mesh
it sounds like what you want to avoid is "telnet character mode":
mesh
In telnet Character mode, only one character is transmitted at a time. The server will acknowledge the receipt of character by echoing it back to the TELNET client. The TELNET client will send back a TCP ACK packet to the TELNET server to inform the receipt of echo from server.
mesh
that level of chattiness would perform terribly over i2p, with one character bouncing back and forth
mesh
In TELNET Line mode, data is transmitted one full line at a time. TELNET client and server negotiate to run in TELNET line mode.
mesh
but in Line mode telnet isn't so different from irc
zzz
remember a tunnel message is 1 KB. say you called flush after every char
zzz
that's 1000x overhead
mesh
zzz: yeah it sounds like the system is more designed for what I think some people call "document based" protocols and not necessarily interactive protocols
mesh
it actually sounds like telnet or even ssh might not be such a good fit for irc. I'll do some more research on this
zzz
our current latency is about 500 ms round trip. there's not a lot you can do to reduce it other than shorter tunnels. if you need sub-100ms latency for your application, it's not going to happen
mesh
zzz: ok. thanks a lot for the tip. I think I'll head back to the drawing board :)
zzz
mesh, easy to test, set up a standard server and standard client in i2ptunnel, point server to port 22 or 23
mesh
foiled by org.apache.http.util haha
mesh
zzz: btw i2p.jar has a package org.apache.http.util ... it's a big problem :/
mesh
hmmm, so the problem is both i2p-1.7.0.jar and httpcore-4.4.15.jar; contains this class org.apache.http.util.Args
mesh
the i2p version of the class is much older and is missing methods that httpcore needs. Fortunately if you move httpcore to appear earlier in the classpath than things sort of work
mesh
the real fun starts if you're app is modularized
mesh
in which case neither of these projects are modules and there's an unrecoverable split package error
mesh
i2p-1.7.0.jar probably should *not* be "exporting" packages in the org.apache* namespace since these are uh apache projects
mesh
might be able to work around this by creating a non-modularized launcher project
mesh
I did this once on an earlier project but the process was so terrible that I think I blocked it out
mesh
I2P's StandardSocket is package private
mesh
but for now we can cut and paste
mesh
and since httpcore uses the ServerSocketFactory we can basically get httpcore's webserver talking to an I2PSocket
mesh
hello world seems to work
zzz
you dont need StandardSocket or anything else in impl
mesh
zzz: does I2P give me an implementation of javax.net.ServerSocketFactory;
zzz
if it's not in net.i2p.client.streaming then no
zzz
what would it bind to?
mesh
zzz: can you take a look at pastebin.com/FBUGTBXs ?
zzz
I2PSocketManager.getStandardServerSocket() gives it to you, it's essentially the factory
mesh
hmmm really
mesh
oh yeah that will work too
mesh
didn't realize that getStandardServerSocket() existed
mesh
I manually did the same thing, creating code that wrapped I2PSocket by copying and pasting StandardSocket
mesh
wrapping that in a ServerSocketFactory and passed it to httpcore
mesh
the good news is that it seems to work. if you go to wndphwm65gnpdmggv5ohai5pasdwstdy7gobnhiks5ag7iwelena.b32.i2p you can see my amazing blog
mesh
the bad news is that both http-core-4.4.5.jar and i2p-1.7.0.jar contain this class org.apache.http.util.Args
mesh
the web server works without creating any kind of i2ptunnel but I think I'm going to have to split the project into a modular and non-modular piece
mesh
I2PSocketManager#getStandardServerSocket works too
mesh
zzz: I think it would still be useful if StandardSocket was public
zzz
no
mesh
unless is there an existing api for making an I2PSocket look like a Socket?
zzz
stay out of impl, b/c it's the impl.
mesh
something like I2PSocket.asSocket() might work. I think in my case I will have a I2PServerSocket that returns I2PSockets and then I would want to feed those into one or more webservers running on different ports
zzz
it's a Socket, that's all you need to know
mesh
zzz: I need to (some of the time) transform I2PSocket to Socket so it can be fed into a web server. In other cases though I want to work with the I2PSocket directly. It all depends on which port the client is accessing.
zzz
no you can't turn one into the other, but if you want to work with Sockets instead of I2PSockets, that's what the "standard" methods are for
zzz
pick one or the other
mesh
zzz: Well I need to support both. Some services want I2PSocket and some services want Socket. It's really just a question of the port.
zzz
unfortunately I2PSocket does not extend Socket, that's a decision made 19 years ago, before my time
mesh
zzz: I mean for now I can create a copy of StandardSocket. That's what I did here pastebin.com/FBUGTBXs. But I think adding a method like I2PSocket#asStandardSocket() would definitely be useful
mesh
at least in my use case I have I2P Services and non-I2P Services all working off the same Destination listening to different ports
mesh
some of them want I2PSocket and others want Socket
mesh
but no I don't think I2PSocket should extend Socket. But an adapter method could be provided
zzz
sure, asSocket() could work, easy enough
mesh
zzz: schweet. also pretty please rename org.apache.http.util. It causes a nasty collision with httpcore :)
zzz
so put your other jars first, or rip it out of i2p.jar
mesh
zzz: I could do that but why not fix it at the source? it's really bad for i2p.jar to export org.apache packages imo
zzz
please file issues on our gitlab for this or other suggestions, state your case there
mesh
zzz: on github?
zzz
no
zzz
git.idk.i2p
mesh
interesting. gitlab over i2p
eyedeekay
Been running it for a couple years, works pretty well
eyedeekay
There's a guide on the site if you ever want to run your own
mesh
eyedeekay: can I host my code there if it's open source?
eyedeekay
Yeah as long as the content is legal in the US and Canada and not spam it's open to everyone
mesh
though of course the sit is mucky for me. sometimes I think I'm the only one in the world who runs windows in high contrast mode
eyedeekay
I haven't done that since like, WinME... Is it the theme being applied by your system interfering with the site or a visual accessibility issue?
eyedeekay
If you need to recolor things, that's actually available to you via your preferences i2pgit.org/-/profile/preferences
eyedeekay
But you would need an account to access that
eyedeekay
/i2pgit.org/git.idk.i2p/
mesh
thanks I'll check it out
mesh
I was thiunking about setting up a gitblit server ( gitblit.github.io/gitblit )
eyedeekay
It would probably be almost the same as gitlab, the thing you as the server operator should watch out for is any user-facing "Import" functionality will need to be disabled or proxied, which can be tricky
mesh
yeah. I'm sure it also leaks personal information like a sieve. though from what I can tell the code is pretty good
eyedeekay
Couldn't speak to how much it leaks exactly, the concern is that with import functionality facing me as an account holder I can tell it to import "https://i2pgit.org/whatever/whatever" and then go into the logs and see what address tried to import whatever/whatever
eyedeekay
Thereby potentially de-anonymizing your service
mesh
yeah good point
eyedeekay
That was a real sonofagun to figure out for gitlab actually, i2pgit's problem was different since it's non-anonymous, I needed to proxy for I2P-to-I2P imports
eyedeekay
And Go honors HTTP_PROXY environment variables, but it doesn't proxy DNS requests for them, and some of the Ruby parts also make requests, so I can't just set up privoxy and set HTTP_PROXY=http://localhost:8118, you have to go in and change settings in multiple places and they're not all necessary or the same
mesh
yeah I can only imagine.the attack surface for something like gitlab and evne gitblit is always going to be huge
mesh
I think ideally you'd want a single git repo exposed via ssh with a read-only website for viewing it
mesh
of course nobody thinks like this any more and sites like gitlab and github to bring a lot of value
mesh
it's just that hosting and securing your own instance is always going to be a pain
eyedeekay
There are services that don't suck to configure, but difficulty rises with complexity
eyedeekay
Gitlab is in many ways a behemoth, if it wasn't monumental to do over again I would have done gitea instead
mesh
eyedeekay: I think what I would eventually like is to be able to do something like: ./i2p-git-server --repo $BARE_REPO_URL
eyedeekay
About gazillion% easier, in fact you can just fork it and stick
eyedeekay
sam, _ := goSam.NewDefaultClient()
eyedeekay
httpClient := &http.Client{ Transport: &http.Transport{ Dial: sam.Dial, },}
eyedeekay
http.DefaultClient = httpClient
eyedeekay
Oh I actually have a thing for that
eyedeekay
It's called "eepHttpd"
mesh
and that would make a single repo available over i2p through ssh+i2p and you could also browse it anonymously though http+i2p
eyedeekay
Yeah that's exactly what it does
mesh
or even git apparently works over http
eyedeekay
I thought it would be a good alternative to github pages for in-I2P projects
mesh
eyedeekay: yeah that's what I want. combined with Destination-based authentication it'd be pretty secure
eyedeekay
But it's clonable too
mesh
you don't get fancy issue management or pull requests or code reviews or release management or CI/CD
eyedeekay
Nothing like that but only because I didn't consider it an important thing to implement at the time
mesh
but it would make it very easy to host git repos over i2p and let some friends collaborate
eyedeekay
That's the idea, pages, git, and SSH on one I2P-based hosting package
mesh
you might not even need ssh but yes it'd be cool
eyedeekay
I would include SSH because it's more familiar to people than encrypted leasesets or authentication but yeah it could be left out too
eyedeekay
But that's how I ended up going on 7 plugins, 3 webextensions and 3 browser-wrappers
eyedeekay
So until I find a way to manage that task a little better, I'm not trying to add more tasks
mesh
haha yeah
mesh
it's why centralization always wins. nobody really wants to run a bunch of git servers if they don't have to
mesh
it's so funny that git, a wonderfully designed decentralized app, ended up being massively centralized heh
eyedeekay
Honestly I think the financial barriers to entry to clearnet hosting and the fact that sites get easier to host as services evolve and not harder gives organizations with human and material resources early advantages that lead to centralization
mesh
there's certainly economies of scale that kick in
eyedeekay
eephttpd should require 20 minutes or less of maintenance a month when it reaches a finished state. Something like that is not a problem to self-host
eyedeekay
There's some overlap here with I2P plugins
eyedeekay
That's what I found to like about them
mesh
see I think it would still centralized.give people a magical button that they can click and they get a git server on their computer that all their friends can access through the magic of i2p... no ned for buying a server, securing it, etc
mesh
they'll still want to use centralized hosting
eyedeekay
Many will. Hopefully a few less
mesh
combine this with the fact that github is free thanks to those economies of scale and it becomes very tempting
eyedeekay
I suppose on a technical level the best thing I can think of to do is close the gap, so to speak
eyedeekay
github is free-and-easy, anything that wants to challenge them, even on a tiny scale, also needs to be free-and-easy
mesh
eyedeekay: yeah. I think people are starting to understand the value in restricting access
mesh
if you could get something close to github but could be severely restricted to the right people
mesh
would be valuable
mesh
man the jpms module stuff in java 9+ absolutely kills me