edict2 multi radical search

How to search the edict2 file for entire compounds matching radicals instead of just kanji.

Continue reading

Posted in edict2, Programming | 3 Comments

word-diff pstree

One thing I mentioned before was the worded diff pstree.

Here is what I use:


touch /tmp/pstree.old

First we want to remove some common daemons from the output that we aren’t interested in. Threads are also not very interesting, so everything with [{...}] should be removed. This could leave some lines with just line characters so those should be removed too.
Finally the diff uses -m0 (apparently it’s faster), -1 for just {+new things+} remove it to see [-removed stuff-] too, and -W to ignore all the whitespace used in pstree -U.
And -w ‘(-’ and -x ‘-)’ are more useful to tell them apart from [ ] which are used for grouping similar named processes in pstree.

pstree -U | grep -v $'agetty\nudevd\natd\ncron\ndbus\ndhcp\nexplorer\ngconf\nservices\nsyslog' | sed 's/[0-9*─]\+\[{[^}]\+}\]//g;/^[^a-zA-Z]*$/d' > /tmp/pstree.new
dwdiff -m0 -w '(-' -x '-)' -W '[] ─┬└│├1234567890*[]{}
' /tmp/pstree.old /tmp/pstree.new

Finally
mv /tmp/pstree.new /tmp/pstree.old

BTW about colorizing in the watch script, to use it on [ and ] you just need to make sure [ is the first one.
colorize='s/\[/&/g;s/[]`>)}]/&/g;s/|/&/g;s/{/&/g;s/</&/g;s/(/&/g'

Screenshot: (window 3)
screenshot_bash_sysmon

Posted in Programming | Comments Off

Color watch

Continuing with http://archlinux.me/procyon/?p=44

I thought adding colors would mess up pr, but if you add the colors after pr, and preserve whatever stood there it will work, because the same amount of characters remain. That means using characters like {} () and color in what is in between.

First to edit watch.c

Before the for-loop add

start_color();
init_pair(49,1,0);
init_pair(50,2,0);
init_pair(51,3,0);
init_pair(52,4,0);
init_pair(53,5,0);
init_pair(54,6,0);
init_pair(55,7,0);
init_pair(56,0,0);
curs_set(0);

Note that 49 is decimal ascii 1

Next the for loop and initializations


for (y = show_title; y < height; y++) {
size_t len = 0;
char *line = NULL;
char *backup = NULL;
char *color= NULL;
getline(&backup,&len,p);
line=backup;
move(y,0);
color_set(0,NULL);

The added part is tokenizing the line and setting attributes


do {
color=strstr(line,"\x1b\x5b\x00");
if (color != NULL) {
color[0]=0;
addstr(line);
if (color[2] == 'm') {
attr_on(COLOR_PAIR(55),NULL);
color+=3;
}
else if (color[2] == '3') {
attr_on(COLOR_PAIR(color[3]), NULL);
color+=5;
}
} else
addstr(line);
line=color;
}
while ( color != NULL);
free(backup);
}

notes: I am pretty sure this is causing a memleak but I don’t know how to fix it (I tried a few things but no success) if you know how, post in the comments please.

That takes care of watch. The script can then use something like


colorize='s/[`|>)}]/&/g;s/{/&/g;s/</&/g;s/(/&/g'

Where  is ^[ : CTRL+V ESC

And for the output just add it at the end

TOPRINTL=$(( $(echo "$PARTB" | wc -l) +1 ))
(echo "$PARTA" | head -n $TOPRINTL ; ( echo "$PARTB" ; echo '`---------------------------------------------------------' ) | sed 's/^$/|/;s/^\([^,|`]\)/| \1/' ) | pr -2 -T -W $realcols | sed "$colorize"
echo "$PARTA" | tail -n +$((TOPRINTL+1)) | sed "$colorize"

Then any ({< will give different colors. For [] it hits recursion or at least something weird, but it should be possible too.

Posted in Programming | 3 Comments

Finding the right CPU stats

In the sysmon I tried

(echo $'MEM\tCMD\tCPU'
ps -Ao rss,comm,pcpu | sort -rn | head -n 10 | sed 's/\([a-z]\) \([a-z]\)/\1_\2/g' ) | awk '{printf "%-9.9s%-15.15s%6.6s\n",$1,$2,$3}'

beyond head it’s just for formatting. (and I get processes with spaces in the filename)

However, as man ps says pcpu is just process’ running time / time it spent in CPU.

Run a process at 0% for 1 hour => 0%
Peak it for 5 minutes and it will go to 1% or something. And not 100% what you really want.

Side tracking a bit… On the forums a while ago there was a post that described how to get total CPU with /proc/stat. I converted it to bash, try it out:

previdle=0
prevtotal=0
while sleep 1; do
eval $(awk '/^cpu /{print "idle=" $5 "; total=" $2+$3+$4+$5 }' /proc/stat)
echo -en '\r'$((100* ( ($total-$prevtotal) - ($idle-$previdle) ) / ($total - $prevtotal) ))
previdle=$idle
prevtotal=$total
done

It will print the current percentage in the same spot.

For a per process usage there is /proc/[0-9]*/stat. It is in the same unit as the above. That means you can take the usage of the interval from above, the previous usage of the process, and use it to get a percentage of that process since the last interval. Kind of like htop.


#! /bin/bash
IFS="
"
previdle=0
prevtotal=0
while true; do
memused=$(free -m | sed -n '3s/^[^0-9]*\([0-9]*\).*/\1/p')

eval $(awk '/^cpu /{print "idle=" $5 "; total=" $2+$3+$4+$5 }' /proc/stat)
intervaltotal=$((total-prevtotal))
echo -e "$memused\tTOTAL\t$((100*( (intervaltotal) - ($idle-$previdle) ) / (intervaltotal) ))" > /tmp/recentcpu.new
previdle=$idle
prevtotal=$total

for line in $(sed 's/ /_/2;s/)_/ /g;s/[()]//g' /proc/[0-9]*/stat 2>/dev/null | awk '{CONVFMT="%d"; print "process=" $2, ";pid=" $1 , ";proctime=" $14+$15+$16+$17, ";rss=" $24*4/1024}'); do
eval $line
prevproc=${array[$pid]:-0}
echo -e "$rss\t$process\t$(( ( 100*($proctime-$prevproc) / intervaltotal ) ))" >> /tmp/recentcpu.new
array[$pid]=$proctime
done
mv /tmp/recentcpu.new /tmp/recentcpu
sleep 5
done

notes: The sed line for the /proc/*/stat allows for 0 or 1 spaces in the process name. The process time is four values, I don’t know if they are the right ones to use, especially the last two. Look at man 5 ps for a definition. The rss value is in pages, for me a page turned out to be 4 bytes, I don’t know if this is standard it’s the first time I heard of it.

It outputs MB memory used (non-cached RSS mem, the good stuff) / process name / cpu% used.

It outputs to a file, so you can run it in the background and get the values from a script. For quick checking you can use:
while true; do clear; sort -rn -k 3,3 /tmp/recentcpu | head -n 10;sleep 5; done

Or with some nice formatting in your own script
(echo 'MEM CMD CPU'; sort -rn -k 3,3 /tmp/recentcpu | head -n 10) | awk '{printf "%-6.6s%-15.15s%4.4s\n",$1,$2,$3}'

I am comparing the results with htop / top / and conky. It is a bit hard to see, but it looks like htop and top are *2, and conky and the above script seem similar. May be a dual core issue? I may look at conky source to see how they get the values.

Posted in Programming | 1 Comment

Bash system monitor

So far as a system monitor I used a heavily edited pyampc. I thought it would be cool to have a system monitor in bash, and separate the mpd stuff. That way I can redo the mpd client in a non idiosyncratic way (have not begun on that), but also because it’s more stable and looks a lot better.

The system monitor is about done now. I was able to keep it fairly simple.

ncurses was necessary, because otherwise you get a tickertype system which looks bad and clear and redrawing is way too slow. So I used a hacked `watch` (procps) that supports unicode.

If you don’t use unicode, then skip this. What I added to watch was: curs_set(0), and the main loop is this, instead of the char by char approach it was before
for (y = show_title; y < height; y++) {
size_t len = 0;
char *line = NULL;
getline(&line,&len,p);
move(y,0);
addstr(line);
free(line);
}

Making it support escape codes is a lot harder, I added color codes, but I end up didn’t using it because it confuses `pr` used below. If you want it let me know.

Then the actual information script.

I wanted two columns to save space. And the following information:
left: worded diff pstree to previous
right: date/freespace/loadavg/ps mem cpu top 10/ clipboards / atjobs / mpd filename and kakasi decoding.

Well which commands you want is different for everyone, if you want to know the above let me know in the comments. So let’s continue with a little example script.

First get the actual amount of columns ($COLUMNS doesn’t seem to work in scripts)
realcols=$(stty size | sed 's/.* //')

Form the left column, PARTA
PARTA=$(pstree -A
date
)

Make sure it is longer than the right column unless you want more work to do. You could add something like
echo $'\n \n \n \n \n \n\n\n\n\n\n\n\n\n\n\n\n\n'
`watch` displays from the top anyway.

Right column:
PARTB=$(mpc)

We need to know how many lines PARTB is because we line it up with `pr`
TOPRINTL=$(( $(echo "$PARTB" | wc -l) +1 ))
Add 1 because we’ll add a line to it later.

Now to print it
( echo "$PARTA" | head -n $TOPRINTL
( echo "$PARTB" ; echo '`---------------------------------------------------------' ) | sed 's/^$/|/;s/^\([^|`]\)/| \1/'
) | pr -2 -T -W $realcols

So we take the first bit of PARTA
we pad PARTB with | and —— to make a small box (make it longer if necessary, if it’s too long pr will cut it off anyway)
Because they are an equal amount of lines we can send it to pr -2 without a problem.

And finally
echo "$PARTA" | tail -n +$((TOPRINTL+1))

Usage:
watch -n 5 -t scriptname

Posted in music, Programming | 1 Comment

Bash loop for monitoring applications

I am trying out exec $0 as an alternative to the while true loop. I am not sure if it’s really a good idea, but the advantage of being able to edit the script while it’s running and see the changes is very appealing to me. If the script doesn’t need variables for the next loop and it doesn’t run often (<1/minute) I think it may be a good idea.

Posted in Programming | 1 Comment

rtorrent display 1 line per torrent

I like to have rtorrent display everything on 1 line instead of 3. Here are some tips how to get this done. I found these to work the same way in both 0.8.2 and 0.8.5

There are two files involved: src/display/window_download_list.cc and src/display/utils.cc

In the first file we change where it calculates the amount of lines to use:

Range range = rak::advance_bidirectional(m_view->begin_visible(),
m_view->focus() != m_view->end_visible() ? m_view->focus() : m_view->begin_visible(),
m_view->end_visible(),
m_canvas->height() / 3);

Simply remove “/ 3″

Next is the part where it calls some functions in utils.cc. Just remove the four lines of the second and third so you just call print_download_title

Now we change the function print_download_title in utils.cc.
Comment out the only line it has:

// return print_buffer(first, last, " %s", d->download()->name().c_str());

Take a look at the other functions to get an idea of what you want to add and how to do that.

Every addition to the line is of the form: first = print_buffer(first, last, printf_format, printf_variables);
At the end ‘first’ is returned.

Here is the entire function I used.

char*
print_download_title(char* first, char* last, core::Download* d) {
// Status
if (!d->download()->is_open())
first = print_buffer(first, last, "stop:");
else if (!d->download()->is_active())
first = print_buffer(first, last, "open:");
else if (!d->is_done())
first = print_buffer(first, last, "load:");
else
first = print_buffer(first, last, "done:");


// Title
first = print_buffer(first, last, "%-52.52s", d->download()->name().c_str());


if (d->is_active()) {
// Size
first = print_buffer(first, last, " %6.0fm", (double)d->download()->file_list()->size_bytes() / (double)(1 << 20));


//Rate
first = print_buffer(first, last, " |%4.0f/%-4.0f| ",
(double)d->download()->up_rate()->rate() / (1 << 10),

(double)d->download()->down_rate()->rate() / (1 << 10));


// Percentage if downloading / Ratio if finished
if (d->download()->is_active()) {
if ( !d->is_done()) {
first = print_download_percentage_done(first,last,d);
} else {
first = print_buffer(first, last, "r:%-3.2f", (double)rpc::call_command_value("d.get_ratio", rpc::make_target(d)) / 1000.0);
}
}
}


// Check whether it is hashing
if (d->is_hash_checking()) {
first = print_buffer(first, last, " hashing...");
}


// Print any tracker messages
if (!d->message().empty()) {
first = print_buffer(first, last, " %s", d->message().c_str());
}


// Fail safe
if (first > last) {
throw torrent::internal_error("print_download_title(...) wrote past end of the buffer.");
}


return first;
}

Posted in Programming | 2 Comments

edict2 Quiz II

I found an interesting site: Kanji Damage

I want to try to make a quiz for the juukugo and verbs on this site. I have not inspected it carefully, but from the slightly chaotic nature it looks like it won’t be easy to get all the information automatically.

What I will try to make is a quiz like this:

Q: I extinguish or erase it.
A: wo kesu
(The reverse would be impossible)

Q: を消す
A: wo kesu

Posted in edict2, Programming | 3 Comments

Roguelikes II

For the gameplay of the roguelike I am thinking of x-com 3.

What I like about x-com 3 is that the maps are really good. I played it recently and found it really boring though, actually I don’t think I ever finished a game of it without cheating. But I think it could work with a faster pacing and smoother squad operations.

I really hate the scifi setting. I like Urban Terror with just rifles.

I have to give multiplayer some thought. I can’t write a story or campaign or AI, and the (nonsensical) gameplay modes of UrT are fun and exciting.

So actually I am leaning more towards ‘UrT with squads’.

Outside of gameplay it will be exactly like UrT.

The gameplay itself will be like x-com but without any RPG elements. Characters are only different from their (free) gear and HP. I am also thinking of removing all chance in combat.

So without the twitch based gameplay from FPS, and without RPG elements, the only thing to set you apart is squad based management and ability to complete the objective (will probably just be bomb mode, CTF doesn’t make sense with a small squad).

Posted in Programming, Roguelike | Comments Off

Music Player I

I wrote an mpd client once. It’s called pyampc and it’s in C.C.

I still use it but I edited it a lot since the last update to suite my needs. So it’s become too idiosyncratic to post updates on. The source code is also a real mess and it doesn’t look very good and the usage is a bit bad.

So I plan to write a new mpd client. Maybe even a non-mpd music player… I like the idea of ordering your entire music collection to symlinks in one large directory. But this will give lots of other problems.

What I have in mind for improving/rewriting pyampc is again an album/artist list and track list, and keeping it to just that. Having as few features as possible. Maybe even have the playlist creation separate.

Pyampc was pretty idiosyncratic from the start and I doubt anyone else has used it. I’d like to change that for this application, and I’m actually not sure if a real minimal application is the right way to go.

Another thing that has concerned me is that for this and pyfmii I didn’t exactly make a nice UI that you expect from ncurses/slang applications like mc. Ncurses can look really good but I’ve been too lazy to try that.

Posted in music, Programming | Comments Off