Current status of the project

Here is what I have been up to till now..

I have my Bootchart prototype finally running.

What all do I have till now?

1. An install script.
2. A shell script that runs instead of init, runs my systemtap script and then calls init.
3. My systemtap script, of course that does the probing for me and gathers the process info and outputs it in XML format.
4. A script that calls my parser and saves the text output ( the renderer is to be built after mid-term evaluation)
5. A java parser that parses the XML output using the SAX interfaces.

The parsed text-mode output looks like:

No of Processes '1249'.
Process Details - Time1214080512744, Pid451, PPid:450, Execname:sh,Probefunc:sys_execve,Pexecname:stapio.
Process Details - Time1214080516148, Pid1, PPid:0, Execname:init,Probefunc:sys_execve,Pexecname:swapper.
Process Details - Time1214080516167, Pid454, PPid:453, Execname:init,Probefunc:sys_execve,Pexecname:init.
Process Details - Time1214080516203, Pid455, PPid:454, Execname:rc.sysinit,Probefunc:sys_execve,Pexecname:rc.sysinit.
......
Note: Processes are not unique.

Now I just need to extend this structure to include the CPU and Disk info.

Another thing I'd like to mention, mounting a temporary file system is necessary even if I log in memory and dump the output in a file at the end of probing. A 'probe end {//do the logging in file }' results in a read-only file-system error
even though you not writing to the file till the system is up.

quick update

okay.. so I coded the XML parser to retrieve the Process information using SAX interface.. (suits my purpose better than DOM)..

I need to modify the renderer to use this now.. and then integrate it to bootchartd.. then 1/3 of my project will be done.. or so I hope.

Writing a mail to my mentor now... will put up details later.

scribbling away

After half a week of trying to figure out and being worried that I'll seriously fall behind, things are much brighter now :)

I have finally figured out what all information is required.
Gathering process information is the simple part. We have pre-defined functions for it:we can probe the sys_fork() and the sys_execve() and sys_exit() (though I still wonder why vfork() and clone() cannot be probed) and use the pre-defined functions there like pid() etc.It was the disk throughput, CPU utilisation part which was a little confusing..I do have some ideas how to implement, not sure if they work though.

1.was thinking of using the processor queue , disk queue for it and use the qsq_ functions. They'll probably point out the bottlenecks very clearly,say if the number of processes waiting is large.This, from the tasks waiting point of view.

2.Or, to look at it from the CPU and disk point of view, we could probe the kernel inline functions like idle_balance etc.

One more thing, one hint that my mentor gave, logging in memory is better option than logging to a file at bootup.

But anyway, my deadline is 11th June so I can very well experiment with these ideas.

I am really happy to finally get to the coding part :) Wrote a script that returns the process information (pid, ppid, execname, pexecname ) whenever a new process is created.Trivial but still... its code!! Once I am able to implement the CPU and Disk info part, I can modify the bootchartd )

p.s Just realised the parent child relationship can be shown easily in the bootchart.. but that comes later in the rendering phase

Follow up on the mail

Thanks Mr. Herald Hoyer for the very prompt reply:)
hmm.. so these ideas go into my TODO too

- possibility to bootchart what happens in initrd
- easily configure the "stop" marker
- automatic image generation with date and time in the name
- determine how fine grained the image is (show only processes with at least x milliseconds, do not show kernel threads)

I'll try to implement them in the best possible manner

progress so far...

The last time I blogged about my project was like a month ago.. so its time for some serious documentation..

Done with the SystemTap tutorials, and halfway through the language reference (further progress will be made as and when required by the project.. as of now... this suffices..).

Done with the git (this and this). The git repository for the Bootchart is here and ill be submitting my patches to my mentor.

Finished reading through/analyzing the current Bootchart code. The results of the analysis phase are as shown above (uploading on blogger for the first time... dunno whether it will be readable...)

Well... I had it bad trying to represent Bootchart in a single root no crosslinks manner (the only structure that freemind supports..as far as I know ) .. and ah yeah... me used it for the first time yesterday :| But its cool...

Hmm.. another digression... I was wondering if there was a way to upload files without actually owning a host or domain...
(had put up this doubt on an IRC... but I guess I made such a pathetic choice of words that it was interpreted as how to upload pics in your blog....not a really bad option but would have loved to have let everyone see the cool folding and unfolding in the .mm files :) and also .. that way I would have been able to use it anywhere.. ) but I was told nopes.. no way.. you cannot do that unless its just text or code.. (in which case I could have gone for pastebin.. thanks General..)

Back to documentation...These are general, non-verified ideas that ill try out and see if they work...

while I was reading that code... I realized that the script directory (that contains bootchart, bootchartd and bootchartd.conf) will be the one that I need to replace using systemtap ... (the conversion from polling to probing happens here).The parsing and the rendering will be conceptually the same.. (maybe I can reuse some portions).. just that ill need to change the format to a standard XML format for which ill have to design the XML format 1st.

Oh and yeah.. my mentor sent me a link to the 30SecondStartup project./me is just mailing them to see if they would like to have some additional metrics showing up in the Bootchart.. will be awesome if this project is of some use to them :)
Looking forward to everything that GSoC would be :)

and /me needs sleep :( signing off ....**yawn**

Oh and I forgot.. GSoC -India informal meet-up will be held on the 8th of June in Delhi :)
Will be terrific..meeting all other GSoC students :)

A thing of beauty ..

They made me realize I am alive after 3 hours of DSP :o

Aaaah now someone help me stop staring at them and get back to work :|

yippie dippy doo :)

I got accepted as a student in GSoC :)
Feels awesome :)

The vivas and the practicals are going fine too ( for a change, phew!)

Hmm.. now that I'm finished with my celebrations.. its time for work now .

Oh I forgot to mention.. I have my Bootchart up ( that was supposed to be the 'update 2' but anyway) and I have been playing around with SystemTap; it is working fine too.

Next, as per the time line I need to learn git, JAXP and start exploring BootChart and figure out what additional metrics can be added. I am supposed to be done with it by the 12th of May.. so run.. run..run.

Installing git now.. and then ill look for some good tutorials on it.