~dr|z3d
@RN
@StormyCloud
@eyedeekay
%Liorar
+Minogami
+RTP
+Titlacahuan
+acetone
+cumlord
+profetikla
+snex
+uop23ip
+weko
An0nm0n
Arch
DeltaOreo
FreefallHeavens
Gid
Hidenet
Irc2PGuest2827392
Irc2PGuest34670
Irc2PGuest53717
Leopold
Nausicaa
Onn4l7h
Onn4|7h
Palimpsest
Unbur
admin
anon2
anu3
cheddah
itsjustme
j6
limak
not_bob_afk
orignal_
poriori
qend-irc2p
u5657
zzz
shiver
Moin, someone else having issues with high joblag after updating to 1.9.0-0? Over the last 5 days i get spikes of 4s loblag sometimes lasting more than 15min. With 1.8.0 avg. lag was 12ms(without spikes), now at 100. SSU2 is enabled, idk if that is the issue.
dr|z3d
hello shiver
dr|z3d
what you running there, i2p or i2p+ ?
dr|z3d
on i2p+, ordinarily you shouldn't see more than around 3-4ms lag most of the time, much less if you're running as a floodfill.
dr|z3d
(which is why I'm asking)
shiver
i2p+ and floodfill
dr|z3d
floodfill enabled: Job lag463 μs
dr|z3d
oh, i2p+.. ok, what's in your job queue that's taking so much time? have a look at /jobs
shiver
the cpu is a bit old, fx4300
dr|z3d
shouldn't be a major issue, I often run my cpu at reduced clock speed to test lower end, and also run on an intel atom tablet.
dr|z3d
how many known peers?
dr|z3d
I ask because things really start to calm down when you're around 4-5K.
shiver
hovers around 6k
shiver
one moment, looking at /jobs
dr|z3d
SSU2 shouldn't affect things, though there are upstream bug fixes in 1.9-0-2+
dr|z3d
you might want to grab the latest dev build and see if that helps.
shiver
Read NetDbavg. 27 ms max. 13 s
shiver
idk what i should look for, nothing sticks out
dr|z3d
is your job lag high right now?
shiver
no, 13ms
dr|z3d
when it is, see what's in the job queue.
dr|z3d
how many just finished jobs are indicated, shiver?
dr|z3d
that == number of threads allocated to the job scheduler.
shiver
4, and the cpu got 4 cores
shiver
i'll try the update
dr|z3d
ok, you can also try allocating a few more threads.
dr|z3d
if you haven't enabled advanced mode, now's a good time. routerconsole.advanced=true in router.config .. but it sounds like you have already.
shiver
yeha i have
dr|z3d
so I'd recommend router.maxJobRunners=8 and see if that improves things.
dr|z3d
I'd also recommend routerconsole.showPeerTestAvg=true for an additional congestion section readout.
shiver
peertest i have already
shiver
do i need to restart for the jobrunners?
dr|z3d
ok, good. you're on top of things :)
dr|z3d
you can also increase the routerrefresh job interval since you're a floodfill.. router.refreshRouterDelay=60000
dr|z3d
that will set the refresh interval at 1m. and yeah, you need to restart for jobrunners.
dr|z3d
> /help/advancedsettings has various things you can tweak if you're feeling adventurous.
shiver
is the dev update zip on skank.i2p the right one?
dr|z3d
yes
dr|z3d
you can configure updates to notify only on /configupdate with the dev url if you want the console to pull it for you.
shiver
oh right
dr|z3d
unsigned updates section, info is there.
dr|z3d
biab
shiver
brb
dr|z3d
anything noticably different yet, shiver? may take some time
shiver
30min uptime and joblag sits at 3ms with short spikes to 8ms, nothing else i notice
shiver
but the issue is the 4s spikes were often 12h apart, so hard to catch
dr|z3d
ok, so that's noticably lower that before?
dr|z3d
also, try adding a job lag graph on /configstats so you can see over time if there's an emerging pattern.
shiver
i have almost all graphs enabled, thats why i noticed the spikes. i2p runs on another machine, i look maybe once a week if something is up xD
dr|z3d
ok
shiver
and i see that with 1.8 the graph is almost flat, no spikes
shiver
right after the update it got bumpy
dr|z3d
ok, that's definitely something to keep an eye on, but it's not behavior I've noticed here.
dr|z3d
maybe allocating more threads to the job scheduler may have smoothed things. time will tell.
shiver
dr|z3d, is there a negative impact if i go to high with jobrunners?
dr|z3d
shouldn't be, not really, the increase ceases to have any meaningful improvement after a while though. if you go super high, then yeah, there will be consequences.
dr|z3d
I think I've tried 128 threads before, router said no.
shiver
:D
dr|z3d
you should be ok on 8, see how you get on with that config.
shiver
thank you for your advice dr|z3d, i'll probably report back tomorrow.
shiver
dr|z3d, small update: i still get the high joblag and looking at /jobs all threads are doing "Lookup LeaseSet & Reply to Client" while taking more than 3sec to complete
shiver
participating tunnels and active peers fall pretty fast at that time
shiver
also, the log for netDb.successTime looks more or less like the joblag one
dr|z3d
shiver: ok, thanks, will see what we can do to mitigate that.
dr|z3d
will have a patched update for you to try in a few minutes that reduces the timeout for lease lookups.
dr|z3d
ok, done and uploaded to /dev/ url.
dr|z3d
reduces timeout from 20 to 5 seconds for lease lookups. should be more than enough still.
shiver
downloading
dr|z3d
what's the spike times looking like?
dr|z3d
(for job lag)
shiver
maybe i make a forum post and ask who else has the issue
shiver
moment
dr|z3d
hold back on the form post for now, you're talking to the developer :)
dr|z3d
*forum
dr|z3d
there's also some upstream work that's about to happen for lease lookups which may also improve things.
shiver
lag was around 600ms for 10min
dr|z3d
ok, not so bad. anything under a second for a sustained period isn't hugely worrying, but I'm sure we can do better.
shiver
well, my tunnels dropped by 700, same with active peers
dr|z3d
ok, that's more concerning.
dr|z3d
I'm looking at the Iterative Search Job paramaters, will tweak those and build another dev update.
dr|z3d
did your update download yet?
dr|z3d
because I'm about to upload a new revision...
dr|z3d
*** uploaded ***
dr|z3d
if / when your upload completes, manually check for updates again and grab that new revision.
shiver
was about to restart
shiver
will do
dr|z3d
much shorter timeouts for the iterative search job and a reduction in the max number of routers to query.
shiver
after the restart i'm head off to bead so don't stress it to much :D
dr|z3d_
no stress.. I'll be testing the patches on a small flotilla of routers.
shiver
brb
dr|z3d
too early to really say, but initial impressions of the latest build are good.
shiver
the joblag number still turns red for a couple sec for me
shiver
in the gui
shiver
65ms lag
shiver
i'm off to bed, thank you for your help!
dr|z3d
ok, sleep well, let me know tomorrow how things are looking o/