zzz
eyedeekay, looks like you had some fights with the news feed this week :)
eyedeekay
Yeah then I realized that it pulls the article summary with a regular expression :P
zzz
I don't know how you ended up putting the whole body of the post in the <summary>
zzz
that won't work
zzz
even if you did parse it right
zzz
it wouldn't display right
eyedeekay
Yeah and when I did it I realized I broke it and had to fix it which is what led me to find the summary thing
zzz
the summary might not even be shown anywhere, or maybe just a tooltip
zzz
don't spend more than 2 secs writing the summary
zzz
looks like you got to the solution after several hours and several checkins
eyedeekay
Yeah I was looking right past it, got the sections confused or something
zzz
if it breaks, the problem is almost always your entries.html, not the generation scripts
lbt
You guys really don't even have a name for that self-written web-site-builder/-server, do you? I still find that amazing/puzzling ;) I mean ... it's sounds a bit as you did an early Hugo as a by-product :P At least when considering that outputting a static version shouldn't be that difficult "what it does already"
dr|z3d
that's a bit vague, lbt. what are you after? a static blog generator?
lbt
I was comparing to Hugo and that is the obvious difference I am aware of
lbt
But also ... if I were to host it, I would be after that feature indeed
dr|z3d
still not sure what you're asking, but for static blog generators, look at lektor or bludit perhaps.
dr|z3d
tor uses lektor for its blog, blog.incognet.io uses bludit.
dr|z3d
(if you need some live examples)
lbt
I'm looking into how things are done in this project. And kind of can't get over how little love/attention this part seems to have
dr|z3d
oh, you're talking about the python scripts?
lbt
Yes, lol
lbt
See what I mean? ;)
dr|z3d
yeah, there's a name. "python scripts" :)
lbt
:))
eyedeekay
If I ever have a kid I'll name them python scripts
dr|z3d
no you won't!
eyedeekay
j/k
dr|z3d
if you ever have a dog, name it anarchy. then you can shout that everywhere.
dr|z3d
:)
dr|z3d
speaking of blogs, you ever get railroad updated, eyedeekay?
eyedeekay
Couple time since we last talked about it IIRC
eyedeekay
But the biggest remaining problem is that I'm building it on too new a system right now, I need to build it against an older glibc or stop using CGO
eyedeekay
ideally both
dr|z3d
ok, let's put that to the test. I fear the worst, but we'll see..
eyedeekay
I doubt it's going to work unless you're using sid
dr|z3d
sid? isn't sid fairly old?
eyedeekay
sid's the unstable one, it's the very new one
eyedeekay
Well, debian new
eyedeekay
Not like, new new
dr|z3d
I thought that was bookworm
eyedeekay
No unstable is always sid, I think bookworm is testing now
eyedeekay
sid's the neighbor kid who breaks your toys
eyedeekay
(Toy Story references, a'la Debian)
dr|z3d
sure, it's all toy story for whatever reason.
dr|z3d
Failed to download plugin from idk.i2p/railroad/railroad-linux.su3
eyedeekay
Hmm, let me check both the site homes, maybe the cron script failed again
dr|z3d
you obviously got the memo about using monospaced fonts for body text.
eyedeekay
Lol at least it's all on one stylesheet now, trying to get this behemoth of a site under control
eyedeekay
But noted, I'll change it at some point
eyedeekay
OK the script is running again, the railroad part should happen soon...
dr|z3d
oh, and re sid, probably lunar lobster is more or less synced.
dr|z3d
assuming railroad works, I don't know why you don't host idk.i2p using that.
eyedeekay
Mostly the enormous incumbent collection of static content, moving it is the bit that sucks
dr|z3d
I don't know how railroad functions, but if it's anything like bludit, you'd just strip the <html> <head> and <body> tags and paste the main content html straight into a post. then worry about the top menu / navigation afterwards.
eyedeekay
Oh I see what's wrong, the version number is configured incorrectly in the script that syncs the sites
eyedeekay
I'm getting rid of this CGO bullshit while it's on my mind and since I've got go-i2pbrowser now
eyedeekay
There, freakin' finally, no more glibc on linux builds
eyedeekay
Forgot I could do that now
lbt
Do I understand correctly that "advanced search" (premium whatever) is required to do a search with like "-filename:*.po"?
lbt
Talking GitLab here, sry
dr|z3d
lbt: you got the repo download locally?
lbt
Ya, using freemiumg grep -v now :o
dr|z3d
lbt: if so, to retrieve a list of all .po files in the workspace, you can run: git grep --files-with-matches .po |grep ".po$"
lbt
Thing was that the po files were cluttering the search results so I wanted to exclude them from searching in the files basically
dr|z3d
git grep != grep
dr|z3d
git grep is about a million times faster.
lbt
Oh, interesting :)
dr|z3d
run it in combination with standard grep to drill down on your search.
lbt
Will check. On this new machine everything seems lightning fast anyways though :o nvme-ssd kinda rocks I guess, especially when you are not used to it yet ;)
dr|z3d
eg git grep "foo" |grep --exclude=bar
dr|z3d
or whatever.
dr|z3d
git grep uses the git db, so it's not running a live search. which is why it's much faster.
lbt
That would still grep all files though, or? Not that it matters, but find | grep -v | xargs grep filters out first. Well, git-grep sounds like worth checking out anways :)
dr|z3d
yeah, all files in the workspace afaik.
dr|z3d
or maybe all committed/added files. you'd have to double check that.
lbt
Need to understand more of git anyways. Messed up a bit earlier
dr|z3d
very handy for finding strings in source code.
dr|z3d
eg git grep "PROP_"
dr|z3d
git's easy to mess up, and once you've got a few key skills under your belt, almost as easy to recover from.
lbt
It's actually what's it's there for even. I just need to understand some more basic concepts (the trees) and things will start to fall together (more) I think
dr|z3d
often useful is the "git checkout ." command, if you mess up your local workspace and want it returned to the state of the last commit (aka HEAD).
dr|z3d
a tree is just the source directory structure. you probably meant branches :)
lbt
Oh really? I ended up doing some ... was it reset HEAD~ or so? And then all of sudden 2 commits from zzz were unstaged there and I went "ooooh ..."
dr|z3d
git stash and git pop also useful.
lbt
Ah, there is worktrees or something? But ya, not understanding underlying concepts already sounds bad ...
dr|z3d
(if you've got a locally modified workspace and want to pull from a repo)
dr|z3d
git reset ~HEAD will more or less achieve the same thing.
lbt
Or even just wording, if tree = worktree :P
dr|z3d
probably
dr|z3d
sounds like tree, worktree and workspace more or less refer to the same thing.
lbt
Ya maybe I had done the pull upstream inbetween or something. checkout . definately is noted for the next time ;)
dr|z3d
if you don't want to lose your changes, git stash before git pull.
dr|z3d
then you can git pop
dr|z3d
(after pull)
dr|z3d
git checkout <filename> will restore an individual file to it's committed state, and you can use wildcard syntax to hit more than one file.
dr|z3d
eg git checkout **/console.css
dr|z3d
that would restore any modified file named console.css to its committed state.
lbt
That's all on uncommitted changes though, right? I had a local commit that I wanted to refactor - then noticed I'm on master, not in a branch. Oh well, I guess you've heard that kind of story before ...;)
dr|z3d
still, if you want to keep your main branch clean and pull from that without issue, just git checkout -b <newbranch> for your local work
lbt
That's what I had forgotten there and only noticed when I was about to push.
dr|z3d
yeah, you want to work on your own branch and then you can commit that to your branch and then merge it with the main branch, cherry-pick the specific commit, or push it to a separate branch and issue a merge request from there.
dr|z3d
obviously to switch between branches you just do git checkout branchname. to see available branches and your current branch, git branch
lbt
branches are supposed to get deleted after a merge then via some gitlab "allow this" magic, right?
dr|z3d
deletion is optional.
lbt
So if I don't select that, I could re-use a branch (name) to work on the same topic further?
dr|z3d
more a matter of taste and working style than anything. do you want multiple branches available on gitlab, or do you want to develop locally in a different branch, push it to the remote repo and then merge with main/master and then delete it? up to you.
lbt
Deleting/not cluttering doesn't sound bad actually
dr|z3d
I do all my work in a branch named dev, with an upstream branch for upstream pulls. when I push to dev, as soon as the commits are merged with master (on gitlab), dev branch is nuked.
dr|z3d
on github, I leave the dev branches there, since github seems to want the entire dev branch uploaded with every commit if the dev branch isn't already there.
lbt
Hm, you do not have a forkin there but are working directly with the i2p.i2p repo, right?
dr|z3d
I have a local dev branch and an upstream branch, upstream gets pulled and then, working in my dev branch, commits get cherry-picked as required.
lbt
Ya. I have a fork (origin) and then local + upstream. One thing more to make a mistake (like pulling my origin instead of upstream when preparing a merge request)
dr|z3d
given the amount of divergence, I can't just pull from upstream into my dev branch.
dr|z3d
so I cherry-pick instead.
lbt
Why are you so divergent? ;) Like many (larger) projects that changed lots but didn't finish and now "temporarily don't update"? Is it something like that?
dr|z3d
no
dr|z3d
just a huge amount of visual and non-visual differences.
dr|z3d
you'd have to run I2P+ to really see what I mean :)
lbt
I haven't even gotten as far as checking what it is albeit seing the name :( I'm good in getting lost in details ;)
dr|z3d
:)
lbt
Much too curious. Don't you trigger me ;)
dr|z3d
well done, eyedeekay, railroad now works.
lbt
Hm, easier to use, better performing, same security. You got some marketing going :)
dr|z3d
:)
dr|z3d
let me know when I2P supports up to 1Gb/s bandwidth share.
dr|z3d
eyedeekay: I say works, the homepage at least loads now, any other page, though, slq error.
dr|z3d
*sql
dr|z3d
ah, the default pages appear to be placeholder links without content is probably why.
lbt
I didn't know there is so much traffic around ;) My routers seem to never reach limits - memory usually goes steadily up till garbage collection (I guess), but other than that there always seems to be free bandwidth for grabs basically
dr|z3d
mostly there isn't, except when the swarm of potentially abusive routers gets going. then I've seen 42MB/s
lbt
Hm, how easy is it to get detailed router stats for monitoring? "icinga-plugin where?" ;)
dr|z3d
> /configstats is your best bet.
dr|z3d
enable advanced stats and away you go.
dr|z3d
eyedeekay: DSA_SHA1 sigtype?!
lbt
Damn, should have done something there 10y ago. Would be interesting to have a look at now. I don't even know how reachable my blog has been :o
dr|z3d
eyedeekay: also, railroad's not correctly registering with the console plugin manager .. indicates it's stopped after autostarting.
dr|z3d
ok, seems to work after a start/stop.
lbt
btw latest I2P version: 2.0.0 - #i2p.chat
dr|z3d
you'd have to mention it there, different channel, different ops.
eyedeekay
dr|zed yeah I've been working my through a bunch of stuff here, going to release a new version in a little while
dr|z3d
ok, eyedeekay, thanks. at least it actually works now \o/
dr|z3d
you probably also want to set the name somewhere so it doesn't appear as "SAM MUX host" or whatever.
dr|z3d
default page is also using google's font api, probably worth neutering that.
dr|z3d
it might also be worth the effort to consolidate the difference OS versions into a single plugin and then just check the OS and run the appropriate binary.
dr|z3d
(or maybe the go runtime a dependency plugin and download that separately)
dr|z3d
*make
dr|z3d
when I need a distraction I might take a deeper dive into the code and see what can be improved presentation-wise.
dr|z3d
plugin description as appears on /configplugins needs to be a lot briefer. help/instructions for deployment in the README, not the description!
dr|z3d
eg "Personal Blogging platform"
lbt
Have timestamps for i2psnark (the log thing on top) ever been suggested/discussed?
eche|off
state report: main system broke down, new hardware ordered, guess until sunday I'll be mostly offline
zzz
:(
eche|off
yeah, unexpected
eche|off
but still reachable on twitter and mail
zzz
any services down?
eche|off
no
eche|off
not the root servers
zzz
ok
eche|off
just my personal private system at home
eche|off
but thats the system I used to manage most time
eche|off
sorry, update to 2.1 donw, after seeing wild spikes from 200k to 4 MB/sec during 1h timeframe
eche|off
inbound limit hit, hmmm
zzz
yup, wild ride started about 24h ago, awaiting bigly update
eche|off
yeah
eche|off
but graphs do show for e.g. 1h time 2 MB/sec out and 4 MB/Sec in
eche|off
on a router with default services, which were mostly even distribute
eche|off
so it is far to much in traffic which did hit the limtis
zzz
that's fine, limits are good
eche|off
sure, but part tunnel count goes to null after that
eche|off
or "near zero"
eche|off
router active peers go down, to
eche|off
so limits DO work :-)
zzz
sure, when limits are hit, we do things to fix it, but shouldn't go to near zero, unless only a few tunnels are hogging all your bw
zzz
check /tunnels
eche|off
na, that was nearly 20h ago, no realtime data
eche|off
3577415183RU gw5o3058214896RU 37-94 Minuten291 MiB918 KBpsTeilnehmer although that one is nice
zzz
ok plus standalone, a little janky but not bad dr|z3d
zzz
why in the world are you doing a XHR for all.css every 5 sec?
zzz
i think bigly flipped their switch about 8 hours ago
obscuratus
I have OutboundEstablishState and OutboundEstablishState2 on Debug log level. A lot of my 'udp.OutboundEstablishState2: new outbound establish to' messages will fill in the 'with address:' portion with ssu://autodetect.not.yet.complete:0, or maybe I have one good ssu address, and the rest look like: ssu://46005023@null:0
zzz
get too close to the TV, all you see is spots
dr|z3d
zzz: build success readout in the sidebar. any more thoughts? :)
zzz
our trend is toward less stuff there, not more
zzz
and nothing the user can do about it
zzz
got a report '5 tunnels using 90% of my traffic', haven't seen that myself, but may be worth a review
dr|z3d
true, there's nothing the user can do to make the build success better, but it's a good indicator of overall network health.
zzz
actually it's not, it's a local view of health. I'd rather they go to stats.i2p for the overall view
zzz
reddit and ilita are filled with q's about what they can do about i2pd build success, let's avoid that
dr|z3d
nothing a post can't remedy. but I take your point. in which case, put it in the congestion section that's only enabled by default in advanced mode :)
zzz
I never look at my own, it's just not useful
zzz
imho if I don't need it nobody else does, but if you disagree, have fun, % = success / (success + reject + timeout)
dr|z3d
lol
dr|z3d
that's an amusing criteria, but thanks.
dr|z3d
re tunnels, I've seen a small proportion using an oversized amount of b/w before, prior to us banning abusive routers.
dr|z3d
5-10 tunnels pulling 2MB/s each. something like that.
obscuratus
dr|z3d: I've been running this patch for a while now. It makes a graph of your local exploratory build success.
obscuratus
One drawback, it gives a number between 0-1000. I couldn't figure out how to store floating point numbers.
dr|z3d
yeah, but does it provide the results as a percentage? :)
zzz
also as soon as you put stats in front of people they start complaining you didn't smooth it right, and you get bombastic PRs like this one github.com/PurpleI2P/i2pd/pull/1833
dr|z3d
haha
dr|z3d
yeah, that's true, zzz. always gonna be complainers. armchair critics.
dr|z3d
obscuratus: stats are stored as a double afaik. in which case, to get an int percentage, you'd do something like (Math.toIntExact(statvalue) / 2)
dr|z3d
* 100 sorry.
dr|z3d
no, / 10.
dr|z3d
if max value is 1000.
dr|z3d
let's see that PR request.
dr|z3d
oh greay, all in russian. looks good though, pictures and everything.
dr|z3d
*great
obscuratus
dr|z3d: Maybe it's out there and I couldn't find it, but I haven't run across a stat that stores in floating point. Can you point me to an example?
dr|z3d
obscuratus: stats are stored as a double. I;m saying for a percentage, an int value is fine.
zzz
the issue with build stats is you have to take the event counts, not the average value, which is a resp. time
zzz
so you get the event counts for all three and then do the math I said above ^^^
obscuratus
Yeah, I use the event count that we already store.
dr|z3d
buildHealth = (1000 * expSuccessEvents) / (expSuccessEvents + expExpireEvents + expRejectEvents);
zzz
and if you wanted to graph it you'd have to make another "derived" stat and update that periodically
obscuratus
zzz: exactly.
dr|z3d
looks like that patch is heading in the right direction, obscuratus, but ideally we want build success for all tunnels, not just exploratory.
zzz
as drz says everything is a double. if you don't like the scale then change it on the way in
zzz
you probably want to multiply by 100, not 1000 :)
dr|z3d
yeah, I was hinting at that. percentage is a better metric. :)
zzz
mystery solved
obscuratus
dr|z3d: Feel free to change that around any way you see fit (if you're interested). I thought I'd throw it out there, but it's not like there's a lot to it.
dr|z3d
thanks, obscuratus, I might see if I can get a nice percentage value for all tunnel builds. feel free to attempt the same in parallel :)
zzz
obscuratus, change from 1000 to 100d, that will force the rest of the computation to double and not truncate
zzz
and declare buildHealth double
obscuratus
zzz: Thanks, I'll give that a look.
dr|z3d
obscuratus: if you want to test adding it to the sidebar, modify the following code to suit and then insert into renderCongestionHTML() on SummaryBarRenderer,java .. the config property is optional, obviously.
dr|z3d
if (_context.getProperty("routerconsole.showPeerTestAvg") != null &&
dr|z3d
_context.getBooleanProperty("routerconsole.showPeerTestAvg")) {
dr|z3d
buf.append("<tr title=\"")
dr|z3d
.append(_t("Average time to test a peer (successful / total)"))
dr|z3d
.append("\">" +
dr|z3d
"<td><a href=\"/profiles\"><b>")
dr|z3d
.append(_t("Peer test average"))
dr|z3d
.append("</b></a></td><td class=\"digits\"><span>")
dr|z3d
.append(_helper.getAvgPeerTestTimeGood()).append(" / " )
dr|z3d
.append(_helper.getAvgPeerTestTime());
dr|z3d
if (_helper.getAvgPeerTestTime() > 0)
dr|z3d
buf.append(" ms");
dr|z3d
buf.append("</span></td></tr>\n");
dr|z3d
}
obscuratus
dr|z3d: Thanks.
zzz
dr|z3d, 1st line not needed, null == false
dr|z3d
ok, zzz, thanks.
zzz
DataHelper.formatDuration2() might also help for the times
dr|z3d
yeah, maybe, though in this instance ms is fine.
dr|z3d
formatDuration2() would start using seconds and floating points, which isn't what I want there.
dr|z3d
the leading value is the most important, should always be ms.
zzz
ok
zzz
harder if you want "a / b ms" esp. if the a and b scales could be different
dr|z3d
yeah. that's why a static "ms" is fine there. values vary from 300 - 2000 ms, so it's all good.
dr|z3d
plus, there's an override config that sets the peer test timeout, in ms, so having values reported there in anything else could be confusing.
zzz
see formatPair() where we coerce a and b to the same scales
dr|z3d
yeah, gotcha, thanks. works for b/w readouts.
zzz
annoying to get right
dr|z3d
yeah, I bet, but satisfying once you do :)
dr|z3d
speaking of smoothing, I'm currently taking a realtime value / 100, plus a 1m avg / 100, dividing both by 2 and then adding to get a something smoothed value. sane?
dr|z3d
not super important to be 100% accurate. seems to be working fine enough.
dr|z3d
using the 1m average helps to prevent the value from jumping all over the place, given that it drives a percent bar.
zzz
I would just pick a rate and use that alone, don't need anything fancy, we don't do that anywhere else
dr|z3d
yeah, thing is, 1m is too long to give a responsive readout. that's the first thing.
dr|z3d
and all my graphs are standardized over a minute, so setting the rate differently would just screw up the graphs :)
dr|z3d
or can you set multiple, out of order rates for the graphs and always have it use the first one?
dr|z3d
{1m, 10s, ...}
dr|z3d
cpu load averaged over a minute is not particularly useful, hence the mixin with the realtime readout.
zzz
i think it's 1 minite minimum
zzz
how about (currentTotalValue + lastTotalValue) / (currentEventCount + lastEventCount) ? that would give you rapidly changing smooted values
zzz
or is that what you just said
zzz
yeah my way weights the two buckets correctly
zzz
if you want a little decay:
zzz
how about ((2*currentTotalValue) + lastTotalValue) / ((2*currentEventCount) + lastEventCount)
dr|z3d
I can try that, see if gives me superior results. thanks. if you want to try for yourself, code's up on gitlab.
zzz
your way vastly overweights a baby current bucket with only a couple samples
dr|z3d
if the stat value = 0, it'll just use the current value instead.
zzz
yeah but if the current event count is 1 and the last event count is 100, you're overweighting the current bucket by 100x
zzz
so thats the opposite of smoothing :)
dr|z3d
ok, I knew it wasn't totally accurate, I'll try your method. the main thing is to reflect cpu load spikes while giving a reasonable average.
dr|z3d
:)
dr|z3d
gitlab.com/i2pplus/I2P.Plus/-/blob/bbc8a9d091632633d115c2ec6dfb87b893301707/apps/routerconsole/java/src/net/i2p/router/web/helpers/SummaryHelper.java#L501
dr|z3d
Rate stat = _context.statManager().getRate("router.cpuLoad").getRate(60*1000);
dr|z3d
long loadAvg = (long) ((2*stat.getCurrentTotalValue() + stat.getLastTotalValue()) / (2*stat.getCurrentEventCount() + stat.getLastEventCount()));
zzz
another sharp "health" spike up but after my premature call saturday night I'm remaining cautious
zzz
yeah thats it dr|z3d, see how it goes, that over a 60-sec rate should get you constant smoothed changes in the sidebar and a nice graph
dr|z3d
thanks, zzz, I'll keep an eye on it.
dr|z3d
the graph is as before, just averaging the value over a minute.
zzz
just calculate and check for zero summed event count so you dont explode
dr|z3d
not sufficient? int load = loadAvg != 0 ? (Math.toIntExact(loadAvg) / 2) + (getCPULoad() / 2) : getCPULoad();
zzz
should be sufficient
dr|z3d
great, thanks. it just falls back to a realtime value if the stats are returning 0.
dr|z3d
what's just as interesting is the potential to throttle or otherwise modify various sub-systems based on the reported cpu load. helps keep the router from not killing the cpu.
dr|z3d
just noticed, I'm still feeding in the realtime value as 50% of the overall value.
zzz
if it were me I'd explicitly calculate and check the denominator before doing the division to be bulletproof, a divide-by-zero is an unchecked exception
zzz
so you can't do a = b / c; if (a == 0) .... // too late
dr|z3d
that's whay the above does.. checks the value isn't zero before doing the division.
dr|z3d
or not?
zzz
you need
zzz
c = d + e
zzz
if (c > 0)
zzz
a = b / c
zzz
else
zzz
a = getcpuload()
dr|z3d
gotcha.
dr|z3d
long loadAvg;
dr|z3d
if ((stat.getCurrentTotalValue() + stat.getLastTotalValue()) > 0 && (stat.getCurrentEventCount() + stat.getLastEventCount()) > 0)
dr|z3d
loadAvg = (long) ((2*stat.getCurrentTotalValue() + stat.getLastTotalValue()) / (2*stat.getCurrentEventCount() + stat.getLastEventCount()));
dr|z3d
else {
dr|z3d
loadAvg = getCPULoad();
dr|z3d
}
zzz
yes, that's _almost_ bulletproof
zzz
because the event counts could get zeroed between calls
zzz
so only call once
zzz
long count = (2 * currentEventCount() + lastEventCount();
zzz
if (count > 0)
zzz
loadavg = ((2*currentTotalValue()) + lastTotalValue()) / count
zzz
_thats_ bulletproof
dr|z3d
alrighty, thanks. :)
dr|z3d
I've added whatever java itself thinks the cpu load is as part of the calculation, zzz. seems to work ok.
dr|z3d
long loadAvg;
dr|z3d
long count = (1 + (3 * stat.getCurrentEventCount() + stat.getLastEventCount()));
dr|z3d
if (count > 1) {
dr|z3d
loadAvg = (long) (getCPULoad() + ((3 * stat.getCurrentTotalValue()) + stat.getLastTotalValue()) / count);
dr|z3d
} else {
dr|z3d
loadAvg = getCPULoad();
dr|z3d
}
dr|z3d
int load = Math.toIntExact(loadAvg);
dr|z3d
and bumped up the current/previous ratio.