#saltr (irc2p) | I2P+ 2.6.5+ >> i2pplus.github.io | skank.i2p | git.skank.i2p | purokishi.i2p | vituperative.github.io/i2pchat | natter.i2p | notbob.i2p | ramble.i2p | shreddit.i2p | cake.i2p | /msg an op for voice

dr|z3d zzz: re multithreading for checking, your original post -> pr3t6cjlcth36djl3qgnfaollcvcs2kbzigzon6derjzjghttp4q.b32.i2p/posts/20456

dr|z3d ~35 new/updated avatars.

not_bob NIce

dr|z3d hopefully they amuse someone other than me.

not_bob I like mine!

not_bob A battle van!

dr|z3d orginal was tickled.

dr|z3d something about that size of image that can really make an impact done right, in the right context like a forum.

dr|z3d and yeah, your van, wouldn't look out of place in gaza.

not_bob Hahaha, yeah.

not_bob I have no desire to visit that location.

not_bob I don't expect americans are welcome there.

not_bob And, none of the languages I know would be useful.

dr|z3d I don't think airbnb are running any promotions over there right now.

not_bob That is sad.

not_bob But, I imagine it's cheap to buy used property.

dr|z3d I don't think there's any left.

not_bob Bah, I need to compile the new i2pd trunk.

not_bob Fair point.

dr|z3d inline images working on zzz's mirror site: pr3t6cjlcth36djl3qgnfaollcvcs2kbzigzon6derjzjghttp4q.b32.i2p/posts?page=35

not_bob Hah, I can't look. My local i2p isn't working.

dr|z3d zzz hasn't picked up the ball yet, I think he's pretending he didn't notice.

not_bob Heh

dr|z3d maybe I need to give him a naked dancing man avatar. then he'll notice. *grin*

not_bob ooooo ;)

dr|z3d kidding, obviously. :)

not_bob Oh, I know.

dr|z3d his avatar's one of my favorites.

not_bob Though, honestly. I don't think i2p will run on a real 386.

not_bob I like idk's icon :)

dr|z3d cleaned it up, removed the background, enhanced it.

not_bob *nod*

not_bob It looks good.

not_bob Tonight I'm going to get my stats together.

dr|z3d you'll be able to do all that fast soon, no?

dr|z3d your card on the way?

dr|z3d apu..

dr|z3d amd..

dr|z3d accelerator...

not_bob I've found some, have not bought one.

dr|z3d get on with it!

not_bob But, I'll do so.

not_bob I know!

not_bob Anyway, tonight is notbob jump stats for end of year.

not_bob I wrote a script to do most of the work, I just need to find it and remember how to use it.

dr|z3d make it look nice. put it on an svg graph or pie chart or something.

dr|z3d joking, probably doesn't merit that treatment.

not_bob It does not. It will be a simple list.

not_bob Ahh, now I remember how it works!

not_bob It's silly.

not_bob Anwyay, I'm going to get on with it.

not_bob Shit, about 90,000 jumps in the last year~!

not_bob Wait, no.

dr|z3d wait, no, 90000 billion!

dr|z3d most of them originating from mars.

not_bob 564,000 jumps

not_bob The 90,000 was the previous year.

not_bob Well then.

not_bob And this number may not be quite right. I only ran it on one instance and * number of instnaces.

not_bob I'll have to run this on every instance and then do more math.

dr|z3d sure, math ain't hard. basic addition.

RN deploy script to each, put results in file, rsync back to one and add them up

orignal question

orignal why do we need to separate encryption and singing keys in identity

orignal like signing key can't exceed 128 bytes?

orignal if we have ECIES+P521 combination why signing key can't be 132 bytes wihtout moving these extra 4 bytes to the certificate?

orignal also applied to future long keys

not_bob RN: Yep. Though, I've got it compiled at this point. It looks nothing like I expected.

not_bob direct.i2p is the top site?

not_bob And it's dead.

not_bob A lot of the top jumps are dead.

not_bob I need to do some more work to only put in living sites.

not_bob Hmm, no. Some sites that were living this year are now dead. Argh, two lists!

RN people look even harder for a cool site that is gone than ones that are still around

not_bob Very true.

not_bob But, my jump service very clearly tells you that it's dead.

RN it's abscence creates more of a mythos around it

not_bob Fair

RN ford the count you are getting include if someone uses a jump link from your index, or just when the router triggers the jump page due to not having a site in address book?

not_bob Almost 500 requests for the site fakeboobs.i2p!

not_bob Only when a router doens't have it in the addressbook.

not_bob If someone uses a link from my index I will never know.

not_bob I don't use a refer link (though, I could) to track what people click on in the list.

RN fakeboobs? was/is a site?

RN just mentioning it probably added a dozen more hits to yer jumper

not_bob I decided several years ago not to do that.

not_bob Also, i was wrong on fakeboobs, read that wrong.

not_bob But, still.

RN LOL

RN I like that you don't do google like tracking things such as referrer links

not_bob Yep

not_bob That site does not exist, and never has.

not_bob lol, yes.

RN the thing that makes tracking nasty in general is that on the outernet you have someone else do it for you

RN so of course those offering the services consolidate the datas

RN and monitze it

not_bob Yep, exactly. All your clicks get logged and sold. I'm not cool with that.

RN most of us here probably feel much aligned with that

RN er.. rather... i mean... "yah, same"

not_bob Shit, do the logs in jetty rotate out?

not_bob It looks like they do.

not_bob Bah

not_bob Well, the hosts.txt stats will be an average then, argh.

zzz yes orignal if we had a new sigtype with long keys, we could say you can't use it with ElG enctype, and put up to 352 bytes in identity

zzz not in the spec because we haven't needed to yet

zzz dr|z3d, yes I'm ignoring you, I am not taking any changes

dr|z3d just north of 60 new avatars now, for the lulz as they say.

orignal how about P521

orignal ?

orignal if cert doesn't have extra length for it it means it inside padding

zzz yeah we could, that's not the way it's defined now, it's only 4 extra bytes and nobody's using P521 so it doesn't matter much

zzz basically the layout is defined for all current enctype/sigtype combinations. If we add new ones, we'll document the layout in the common structures spec, as usual.

dr|z3d in other news, it appears that around 5-6K possible attack routers have fallen off the network.

dr|z3d any timeframe for the full keepalive implementation to land in the MR tray, zzz?

zzz "early january" but see my roadmap for dependencies and updates. Awaiting review of MR 166

zzz did you take the news status and susimail speedup MRs yet, if you're looking for something to merge?

zzz so there's lots of other MRs up, both approved and not, for you to play with now. keepalive will be up after I've really beaten on it, no time yet

dr|z3d yup, think I got both of those MRs, the more weighty ones I'm waiting for them to be merged, they're a bit unwieldy to do manually.

dr|z3d or maybe I haven't got the news one yet, but susimail fo' sure.

dr|z3d susimail loading, good stuff, just reading the headers seems sane.

dr|z3d is there any value caching the mail index to disk and updating when new mails arrive?

zzz the current file system storage is the disk cache. the headers are in memory

dr|z3d yeah, sure, I'm asking if it's worth caching an index of the headers once they've been read to speed up subsequent loading.

zzz maybe, but it's fast enough for now

dr|z3d up to you, just an idea.

zzz I'm testing on about 4500 emails totally a half a gig

dr|z3d and how long is that taking from login to inbox display, roughly?

zzz couple seconds

zzz vs. 90 sec before

dr|z3d tolerable :)

zzz the real place where we'd need caching is for mail body search, but I don't know how to do that

zzz the search I have now is sender+subject only

zzz can't scan half a gig for every search

dr|z3d you don't know how to search the body, or you don't know how to cache the results?

zzz don't know how to do it quickly without a cache, and don't know how to do a cache, so I'm stuck

dr|z3d basically you want an efficient index you can search, which is updated when new mails arrive. how we get there, not sure.

dr|z3d let's see if there's anything out there that can make things easy..

zzz I don't work for google so I know nothing about search caching

zzz how would I make a search index of a half a gig of emails that isn't another half a gig?

dr|z3d ok, here's your starter: lucene.apache.org

zzz oy no

dr|z3d oy no to you too :P

zzz but presumably most of the half gig is attachments, so that's a start

dr|z3d this is the meat of the project: lucene.apache.org/core

dr|z3d > index size roughly 20-30% the size of text indexed

zzz and somebody has to parse the html emails before indexing

zzz but yeah, if e.g. 90% of the size of the email corpus is attachments, and half the rest is html nonsense, 5% is plain text, an indexer cuts that in 4, leaving the index as 1% of the corpus size

zzz in theory

zzz but yes, lucene is the mac daddy

dr|z3d you can parse html with something like jsoup to get at the text.

dr|z3d not sure if lucene is overkill, maybe you just need solr.apache.org

dr|z3d I say "just need", it's built on top of lucene so ignore that part.

zzz it's just far outside my comfort zone atm, not saying it wouldn't be a fun project some time

dr|z3d if you did decide to take that on, it'd probably be useful globally, not just in susimail.

zzz where else?

dr|z3d and not just in the router, either. conceivably it could be used on the website.

dr|z3d where else, all over the router, for a start.

dr|z3d think of firefox's config search, for example.

zzz that's console, not router. where would router need text search?

dr|z3d you know, where's the wotsit for configuring floodfill, that sort of thing.

dr|z3d I meant the console, sorry. it's user facing, so not so much in the router per se, unless you include looking for dests, RIs, that sort of thing.

zzz console index could be an offline solution, doesn't have to be java

zzz ditto website

dr|z3d true enough, just a question of leveraging the library if you decide to use it to do more.

zzz yeah it's just two different problem spaces. might have similar solutions, might not

dr|z3d global search in console and webapps for a dest, or partial dest, for example, that could be handy.

dr|z3d well, it's worth thinking about, if you decide to roll with it, then the scope and usefulness will become clearer for different contexts.

zzz call that susimail search phase 3, after GET search and XHR search, ETA 2025

dr|z3d :)

T3s|4 zzz: they don't describe the process steps, but protonmail (somewhat recently) supports content search. The key seems to be d/ling an index. On some of my larger Inboxes, that's taken 30+ mins. Process overview: proton.me/support/search-message-content

snex in snark: create new torrent, in "data to seed" put /some/file/here. then go into settings and try to change your default data dir. that torrent you just made gets hosed

snex specifically it will lose the fact that you are seeding from a location and it will start trying to download to that new location from scratch

dr|z3d hey snex

snex hi

dr|z3d finally, welcome to the channel :)

dr|z3d if changing the data dir is messing with torrents outside the data dir, that's something for zzz to look at.

zzz yeah, don't do that

snex wish i knew this about 30 mins ago

zzz we could prevent changing data dir if any torrents are running (if we don't, not sure) but even that wouldn't help you here

snex i just lost like 400 torrents of my home videos, linux isos, and other completely legal files

zzz hmph

snex i feel like it should remember that a given torrent was created with a given data dir. it seems to survive router restarts just fine

zzz not sure exactly why

snex i have kind of a unique setup going on here. my data is all on a NAS NFS share. my i2p is running on a rasp-pi-like device that mounts this NFS share

zzz the full absolute path to the data dir is stored in the per-torrent config files

zzz you can see it in ~/.i2p/i2psnark.config.d/*/*.config files

zzz it's not relative to the data dir

snex well ive already wiped everything and am rebuilding so i cant say what was in those prior

snex it does look fine for the ones i have already re-entered so far

snex does changing the default dir update these files?

zzz no

zzz the actual data files got corrupted as it started to re-download each one? or just the .torrent files or the config files got changed?

zzz if your .torrent files got renamed to .torrent.BAD you can rename them back

snex the data files are fine, but it started trying to re-download each one into the new data dir rather than recognzing they were being seeded

snex there were no torrent.BAD files anywhere

zzz were torrents running when you changed the data dir?

snex i had to shut everything down because it was trying to put 1TB worth of data onto a 64G drive

snex yes

zzz ok well that we should probably fix

zzz not sure if that's the cause or not though

snex the only other thing of note is that snark seems to think the data dir i originally tried is not writable

snex so maybe setting to a valid dir doesnt reveal any problems, but setting to an invalid one and then back does

snex hm, so heres a question. if i make a torrent file on my desktop, then also manually make one of these config files and place it there, will snark recognize it? because this will make things so much easier for me

snex whats the significance of these subfolders "s2" "sJ" etc?

zzz you

zzz just so it's not one huge directory

zzz I don't know if it will find the config file

zzz probably not?

snex i would very much prefer to make the torrent on my desktop and then just have snark recognize the data location

zzz you could stop i2p, change the data dir in the snark config file, copy all the torrent files over to the snark dir, and restart

zzz then if you want to change the data dir back, stop and edit the config file again

dr|z3d no need to stop i2p, just stop i2psnark in /configwebapps

zzz should work

snex well my data dir issue is a whole nother thing. it insists it cannot write to a folder on the NFS share when it definitely can

zzz just guessing though

zzz set all your seeded files 444 just in case

snex so this may sound weird but i want the data dir to be something different than my seed location

snex because the seed location is organized in a way that i want, whereas downloaded data is organized in the way other people want. i need to be able to fix it before i put it into my data store

snex if that makes sense

zzz yeah I get it

zzz we don't support move-completed-torrents-somewhere-else

zzz or two data dirs

snex well i dont necessarily want that either because it depends on the nature of the data. i want linux isos to go into "linux_isos" and unix isos to go into "unix_isos" for example

snex "steamboat willie" should go into "public_domain_movies"

snex it would be great if i could do all this stuff with symlinks but nobody likes those :(

zzz maybe see if biglybt will do what you need?

snex it might but i dont want my desktop to be running anything server-like

snex im going to look into XD after i get all these torrents back

zzz XD would be a step back. no DHT

snex bleh

zzz but worth playing with

snex if you need a hand working on any of this stuff i can help out. java isnt my favorite but i can do it

zzz anyway,sorry for your trouble, especially if I had anything to do with it

zzz seems like all your old .torrent files should still be in the old data dir, right where you left them, so I don't get it

snex they were. i had to nuke them. or thought i did and then did

zzz hmm

snex i just didnt know how to make them refer back to the original data locations

snex now that i know about those config files, i should be able to play with them if this happens again

zzz yeah because the torrent file monitor would have blown away all the config files after you changed the location

snex hmm im gonna try something real quick...

RN famous last words

snex cant get into a worse spot than im already in

RN :D

RN well, sure you could. you could loose the data files

snex i have backups

snex or in the case of linux isos, i can just re-grab

RN good good

RN just teasing... cause... you know... 'snex'

zzz as usual, it's the panic after the first problem that dooms you

snex if i didnt panic when i sliced a giant chunk of my hand off with a mandolin, im not gonna panic over a few lost days of productivity by having to re-add torrents

zzz yikes

snex but if what im thinking of works, that time will be shortened by a lot

zzz yeah every disaster has a lesson

snex i just have to wait for this 12G file to add itself first

zzz I have some ideas on how to make it more bullet-proof, adding to the list

snex snexs_12th_birthday_2160p.mkv

snex i bet you didnt know they had 2160p cameras back in 1994

snex damn, didnt work

snex i was thinking to upload the .torrent file that i already have, then stop it, change the configs, then start it back up. but that doesnt work

snex i might have to stop the whole app for this to work

zzz yeah in the best case, if you switched your data dir, didn't delete your torrents, realized the issue, and switched back, ...

zzz it would still delete all the torrent config files and then would have to recheck every torrent when you switched back

snex well i switched back first. the problem is the original data dir was already not where the data is

zzz oh then it would have created all new empty data files

snex yup

snex it tried to create 1TB of empty files in 64G of space

zzz bottom line, if there's no .torrent file it's going to delete the .config file

zzz yup

zzz but yeah, you could have stopped snark, fixed up all the config files, and restarted. But if it's yacking on out-of-disk then it gets harder

snex so like heres my setup, i have my data in /mnt/data/live_data. my i2p datadir is /var/lib/i2p/i2psnark. i WANT my datadir to be /mnt/data/staging_data but snark insists that it cant write here, so i had to set it back

zzz dunno, we just use java to tell us if a dir is writable

snex yeah. i have no idea why it insists it cant write here because sudo -u i2psvc touch /mnt/data/staging_data/wtf works just fine

zzz anyway, as long as you still have the data files, it's all recoverable, not a mandolin type situation ))

snex eh, my hand healed eventually