Mid term evals!! eeks!

Quoting from my previous post "Now I just need to extend this structure to include the CPU and Disk info."

Well.. it turned out to be anything but just.

For 1, as I am not to use /proc files, that rules out passing those values to my systemtap script using command line arguments.
So, we decided to look into the kernel code and see how those entries are filled in the first place so that we could tap the information from there itself.

From there, I figured out that the files that particularly interest me are
1. /proc/proc_misc.c
2. /proc/array.c
3. /block/genhd.c

Then I mailed the systemtap mailing list and they pointed me to this. It is a very interesting tapset but the following bits left me a little unsure about using it-

Note that blktrace needs to be running in order for these scripts to
have any effect - __blk_add_trace() and therefore the tapset probe
isn't actually called unless tracing is active. Of course, that means
that you need to enable CONFIG_BLK_DEV_IO_TRACE in the kernel.


Things works fine this way, if you're careful - for some reason, if you
don't define overrides for *all* the callback functions, *none* of them
get called. The same thing is apparently true wrt optimization - if any
one of the callback functions gets optimized out, they all do. So in
your script, you need to define handler functions for every event type
whether you use it or not, and furthermore the bodies of the unused
handlers need to contain code that won't be optimized out

Installing blktrace sounded like adding additional baggage to us. Also, I haven't tried out blktrace so I do not know how to use this and whether using it will solve my purpose. I could of course play around with it for sometime but mid term evaluations start on 7th so I probed sys_read and sys_write to count the total reads and writes, and then used guru mode to find out the number of I/O tasks pending at the moment.

Yeah , I know guru mode needs to be used only when there is no other alternative . I couldn't see any. Any suggestion would be very welcome.

Thankfully, it works and I have all the diskstats that I require.So far so good.

But CPU information is turning out to be trickier. The variables that I require are not accessible.

A stap -p2 -e 'probe kernel.function("do_task_stat") {$foo}' -u ( A trick I learnt from fche )
gives the alternatives as : task buffer whole vsize eip esp wchan priority nice tty_pgrp tty_nr sigign sigcatch state res ppid pgid sid num_threads mm start_time cmin_flt cmaj_flt min_flt maj_flt cutime cstime utime stime cgtime gtime rsslim tcomm flags ns

But the variables utime, stime are defined within the function and a
stap -p2 -e 'probe kernel.function("do_task_stat") {$utime}' -u
gives semantic error: not accessible at this address: identifier '$utime' at < input > :1:40

I guess the only variables accessible are the parameters that are passed to the function and the return value as the probe is places at the start of the function definition. and the variables I need have their values worked out inside the function and they get seq.printed from there itself .. so these values are not even returned.

The kernel.statement construct is not what I can use as absolute addresses may be different on different computers.

As of now .. I am trying a very very ugly solution..

Edit 1: License stuff :D
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 of the License.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.


%{
#include< linux/kernel_stat.h >
#include< linux/sched.h >
%}

function get_str_info:long(val:long)
%{
struct task_struct *temp = (struct task_struct*)(long)THIS->val;
long x =(long)((temp->state & (0| 1 | 2 | 4| 8))| temp->exit_state);
THIS->__retvalue = x;
%}

function get_u_info_1:long(val:long)
%{
struct task_struct *temp = (struct task_struct*)(long)THIS->val;
cputime_t ut;
long usr;
struct signal_struct *si = temp->signal;
struct task_struct *t = temp;
do {
ut = cputime_add(ut,t->utime);
t = next_thread(t);
} while (t != temp);
ut = cputime_add(ut, si->utime);
usr = (long) ut;
THIS->__retvalue = usr;
%}


function get_u_info_2:long(val:long)
%{
struct task_struct *temp = (struct task_struct*)(long)THIS->val;
cputime_t ut;
long usr;
ut = temp->utime;
usr = (long)ut;
THIS->__retvalue = usr;
%}

function get_s_info_1:long(val:long)
%{
struct task_struct *temp = (struct task_struct*)(long)THIS->val;
long sys;
struct signal_struct *si = temp->signal;
struct task_struct *t = temp;
do {
st = cputime_add(st,t->stime);
t = next_thread(t);
} while (t != temp);
st = cputime_add(st, si->stime);
sys = (long) st;
THIS->__retvalue = sys;
%}

function get_s_info_2:long(val:long)
%{
struct task_struct *temp = (struct task_struct*)(long)THIS->val;
cputime_t st;
long sys;
st = temp->stime;
sys = (long)st;
THIS->__retvalue = sys;
%}

probe kernel.function("do_task_stat")
{
if ($whole)
{
u = get_u_info_1($task)
s = get_s_info_1($task)
state = get_state_info($task)
}
else
{
u = get_u_info_2($task)
s = get_s_info_2($task)
state= get_state_info($task)
}
}

Of course the above code needs to be debugged for successful compilation : function get_state_info:long works but the rest need some more work ( probably due to the calls to other inline functions: next_thread()and cputime_add() within the function ). The reason I am putting up the unfinished code is because I need to know whether its worth spending time debugging it.Is it the right way to proceed?

What I am basically trying to do is.. take the values of task and whole (they are passed as parameters to the function do_task_stat and hence are available. From there I calculate the variables I needed the same way as is being done inside the original function. *very ugly* but am at a loss of ideas (for now at least).

Anyway, I'll just work on this today and tomorrow. If I am unable to find a better way, I guess I'll move on to the cleaning up tasks so that my Bootchart is perfectly ready with the process and disk info at least.

Then I can come back to this after mid-term evaluations.

Current status of the project

Here is what I have been up to till now..

I have my Bootchart prototype finally running.

What all do I have till now?

1. An install script.
2. A shell script that runs instead of init, runs my systemtap script and then calls init.
3. My systemtap script, of course that does the probing for me and gathers the process info and outputs it in XML format.
4. A script that calls my parser and saves the text output ( the renderer is to be built after mid-term evaluation)
5. A java parser that parses the XML output using the SAX interfaces.

The parsed text-mode output looks like:

No of Processes '1249'.
Process Details - Time1214080512744, Pid451, PPid:450, Execname:sh,Probefunc:sys_execve,Pexecname:stapio.
Process Details - Time1214080516148, Pid1, PPid:0, Execname:init,Probefunc:sys_execve,Pexecname:swapper.
Process Details - Time1214080516167, Pid454, PPid:453, Execname:init,Probefunc:sys_execve,Pexecname:init.
Process Details - Time1214080516203, Pid455, PPid:454, Execname:rc.sysinit,Probefunc:sys_execve,Pexecname:rc.sysinit.
......
Note: Processes are not unique.

Now I just need to extend this structure to include the CPU and Disk info.

Another thing I'd like to mention, mounting a temporary file system is necessary even if I log in memory and dump the output in a file at the end of probing. A 'probe end {//do the logging in file }' results in a read-only file-system error
even though you not writing to the file till the system is up.

quick update

okay.. so I coded the XML parser to retrieve the Process information using SAX interface.. (suits my purpose better than DOM)..

I need to modify the renderer to use this now.. and then integrate it to bootchartd.. then 1/3 of my project will be done.. or so I hope.

Writing a mail to my mentor now... will put up details later.

scribbling away

After half a week of trying to figure out and being worried that I'll seriously fall behind, things are much brighter now :)

I have finally figured out what all information is required.
Gathering process information is the simple part. We have pre-defined functions for it:we can probe the sys_fork() and the sys_execve() and sys_exit() (though I still wonder why vfork() and clone() cannot be probed) and use the pre-defined functions there like pid() etc.It was the disk throughput, CPU utilisation part which was a little confusing..I do have some ideas how to implement, not sure if they work though.

1.was thinking of using the processor queue , disk queue for it and use the qsq_ functions. They'll probably point out the bottlenecks very clearly,say if the number of processes waiting is large.This, from the tasks waiting point of view.

2.Or, to look at it from the CPU and disk point of view, we could probe the kernel inline functions like idle_balance etc.

One more thing, one hint that my mentor gave, logging in memory is better option than logging to a file at bootup.

But anyway, my deadline is 11th June so I can very well experiment with these ideas.

I am really happy to finally get to the coding part :) Wrote a script that returns the process information (pid, ppid, execname, pexecname ) whenever a new process is created.Trivial but still... its code!! Once I am able to implement the CPU and Disk info part, I can modify the bootchartd )

p.s Just realised the parent child relationship can be shown easily in the bootchart.. but that comes later in the rendering phase

Follow up on the mail

Thanks Mr. Herald Hoyer for the very prompt reply:)
hmm.. so these ideas go into my TODO too

- possibility to bootchart what happens in initrd
- easily configure the "stop" marker
- automatic image generation with date and time in the name
- determine how fine grained the image is (show only processes with at least x milliseconds, do not show kernel threads)

I'll try to implement them in the best possible manner

progress so far...

The last time I blogged about my project was like a month ago.. so its time for some serious documentation..

Done with the SystemTap tutorials, and halfway through the language reference (further progress will be made as and when required by the project.. as of now... this suffices..).

Done with the git (this and this). The git repository for the Bootchart is here and ill be submitting my patches to my mentor.

Finished reading through/analyzing the current Bootchart code. The results of the analysis phase are as shown above (uploading on blogger for the first time... dunno whether it will be readable...)

Well... I had it bad trying to represent Bootchart in a single root no crosslinks manner (the only structure that freemind supports..as far as I know ) .. and ah yeah... me used it for the first time yesterday :| But its cool...

Hmm.. another digression... I was wondering if there was a way to upload files without actually owning a host or domain...
(had put up this doubt on an IRC... but I guess I made such a pathetic choice of words that it was interpreted as how to upload pics in your blog....not a really bad option but would have loved to have let everyone see the cool folding and unfolding in the .mm files :) and also .. that way I would have been able to use it anywhere.. ) but I was told nopes.. no way.. you cannot do that unless its just text or code.. (in which case I could have gone for pastebin.. thanks General..)

Back to documentation...These are general, non-verified ideas that ill try out and see if they work...

while I was reading that code... I realized that the script directory (that contains bootchart, bootchartd and bootchartd.conf) will be the one that I need to replace using systemtap ... (the conversion from polling to probing happens here).The parsing and the rendering will be conceptually the same.. (maybe I can reuse some portions).. just that ill need to change the format to a standard XML format for which ill have to design the XML format 1st.

Oh and yeah.. my mentor sent me a link to the 30SecondStartup project./me is just mailing them to see if they would like to have some additional metrics showing up in the Bootchart.. will be awesome if this project is of some use to them :)
Looking forward to everything that GSoC would be :)

and /me needs sleep :( signing off ....**yawn**

Oh and I forgot.. GSoC -India informal meet-up will be held on the 8th of June in Delhi :)
Will be terrific..meeting all other GSoC students :)

A thing of beauty ..

They made me realize I am alive after 3 hours of DSP :o

Aaaah now someone help me stop staring at them and get back to work :|