Postscript flowchart symbols for graphviz

I've been wanting a more complete set of flowchart symbols for graphviz for a long time. Since today is a holiday and the kids are out of town I had some time to finally hack up some postscript flowchart shapes that graphviz can use. Some shapes are already available but others have been missing. With the above shape file, you have to use the postscript driver to have access to a more complete set of flowchart shapes. But once you've generated postscript, you can convert the output to other image formats with something like ghostview. Here is an example - a student registration process as a flow chart. This example was proposed as a swimlane flowchart and originally from an article in CA magazine.

Graphviz flowchart example: A Student registration process

Scripted vmdk/ova images w/boxgrinder and virtualbox

It wasn't immediately clear to me from boxgrinder or virtualbox documentation how to script the complete creation of a self-contained runnable virtual image. Since it took me some effort to get it all together, I'll post here a complete example. The script creates a Fedora 16 w/graphical UI machine image by first creating a virtualbox vmdk image (pretty straightforward) and then from that, scripted creation of a more self-contained .ova image (less straightforward).

I have the following packages installed:

• VirtualBox-OSE
  
• rubygem-boxgrinder-build
  

Starting from a boxgrinder appliance definition file...

http://boxgrinder.org/tutorials/appliance-definition/

...the script creates an .ova image that can be easily imported and used by 3rd parties like this:

  $ VBoxManage import f16-xfce.ova
  $ VirtualBox --startvm f16-xfce

ministat example for comparing performance

This is mostly a reminder to myself how to get and use FreeBSD's ministat for performance comparisons. This somewhat useless example shows that -server is apparently the default flag for java these days. Note: you have to strip out some FreeBSD-isms to get it to compile on Linux.

Cloud abstraction APIs getting interesting

With many people rightly nervous about compute cloud lock-in, we seem to be entering an interesting time in the development of compute cloud abstraction libraries and APIs for trying to work around this. I noticed that last month RedHat's deltacloud has become the apache deltacloud (Ruby) incubator project alongside the already existing apache libcloud (Python) incubator project, which has a pretty impressive list of supported providers. An interesting alternative to these is jclouds (Java) with its own impressive list of supported providers. However, just this month, libcloud has announced that they too are working on a Java version of their API.

According to Ohloh, all of the libcloud, deltacloud, and jclouds projects are showing a lot of recent development activity - very good news for those of us rooting for the possibility of lockin-free compute clouds.

Learning how to teach physics (and probably math)

Eric Mazur, a Harvard professor, talks about his experience discovering that in spite of good in-class exam scores and good end-of-the-semester teacher evaluations, his students shockingly weren't internalizing elementary physics concepts e.g. in the context of everyday situations. He had only learned this because he had seen the result presented at a conference and didn't believe it, so decided to test his own classes to disprove the result. He was shocked that it was true in his classes as well - year after year.

He could have left things as-is - all the standard metrics (exams, evaluations) showed there was no dire need to change anything. But true conceptual internalization wasn't taking place.

Where was the problem? Wasn't he a good teacher? And weren't his students of the highest quality?

Dr Mazur discusses the characteristics of the concepts he wanted his students to internalize, the tests he used to test them, the metrics to measure the results and his multiple attempts to change his teaching style to achieve those results.

The process that in the end produced good outcomes (and has been validated in programs throughout the country) turns out to focus less on lecturing and more on directing peer-instruction. Well worth a watch...

Eric Mazur presents Confessions of a Converted Lecturer

There seems to also be good evidence for peer-assisted instruction for elementary level mathematics as well.

What makes a great teacher?

What makes a great teacher? Probably not what you think. Here is an article in the Atlantic describing data collected and analyzed by Teach for America (the non-profit organization that recruits recent college graduates and professionals to teach for two years in low-income communities throughout the US).

Starting in 2002, Teach for America began using student test-score progress data to put teachers into one of three categories: those who move their students one and a half or more years ahead in one year; those who achieve one to one and a half years of growth; and those who yield less than one year of gains.

According to the article, the characteristics could be summarized as:

set big goals

continuously seek to improve effectiveness

recruit students and families into the process

plan exhaustively and purposefully

work backwards from desired outcome

refuse to surrender to [...] bureaucracy and budgetary shortfalls

At the moment, TfA claims larger gains than can be independently verified (which are also only for math) but their approach still looks promising...

So far, only one independent, random-assignment study of Teach for America’s effectiveness has been conducted. That report, published by Mathematica Policy Research in 2004, looked at the organization’s teachers and found that, in math, their students significantly outperformed those of their more experienced counterparts. (In reading, though, the teachers’ students did the same as other teachers’ students.) Another study is due out in 2012 or 2013.

For the teaching of reading, Siegfried Engelmann's Direct Instruction seems to have years of evidence. And you can purchase an e-tutor that implements the method for home use

web and print with moinmoin, pdflatex, google charts and eastwood

As always, this is mainly a reminder to myself... We wanted to create a nice report/handbook that will be occasionally be printed, but in between printings should be a living (web) document. As typical for reports, it will contain charts and tables. We went with a wiki, which has pretty good tables and google charts looked like a perfect fit, until we saw that output is limited to png with a maximum resolution of 300k pixels. Then we found eastwood, a reimplementation of the google chart api in java with jfreechart as the back end.

Poor Man's Java memory profiler

Here is an example of do-it-yourself memory profiling e.g. showing used heap memory over the life of a program run. I compared it with results from YourKit to convince myself it's ok.

The concept:

• Launch a separate thread that collects used memory stats
  
• add a shutdown hook to output the stats

JUnitPerf timed load JUnit test example

For my own future reference, here is a complete example of a JUnitPerf timed load test of a JUnit test.

First the normal JUnit test case.

A perl-wrapped java-based Nagios MS SQL plugin: check_mssql

I didn't find a nagios plugin that I like for monitoring our MS SQL databases. So I hacked up one in java using the jtds jdbc driver. Yes, the 2 minor things I do to improve "security" are horrible. I'll handle those things better later....

check_http as a Nagios Web Service plugin

We typically use nagios to monitor our hardware and hosted services. Now that we are starting to offer more web services, we wanted to replace the ad-hoc curl wrapper we were using with a more standard nagios plugin. After searching for one and not finding any, it occurred to me that if curl can do it, maybe you can trick the standard check_http plugin to also do it. A quick experiment with an https hosted web service (self signed certificate for this test) shows that it works:

./check_http -S -k 'SOAPAction: ping' -r 'Current Status.*OK' -T text/xml -P '<s:envelope s="http://schemas.xmlsoap.org/soap/envelope/"><s:header/><s:body><ping/><s:Body><s:Envelope>' -t 10 -m 512 -p 8083 -H 192.168.1.254 -u /Services/Search/search.svc
HTTP OK HTTP/1.1 200 OK - 0.033 second response time |time=0.032722s;;;0.000000 size=897B;512;0;0

Now time to look at nagios grapher...

Siebel COM programming with Groovy

After my success in using Perl to program the siebel COM interface, I thought I'd try groovy. Yes it works as well.

Siebel COM programming with Perl

I wanted to play around with the Siebel COM interface using a dynamic language instead of having to re-compile after every change. I happened to have perl installed on my machine so it was the natural choice. Since google didn't show me any usable hits beforehand and it took a bit of investigation, I'm posting my results here.

Csharp checkstyle partial solution

There doesn't seem to be an equivalent to Java/Eclipse's checkstyle for Csharp.Net

For the purpose of attempting to enforce a common code formatting style, microsoft recommends a per-solution config file together with a VB macro that automatically loads it up when you open the solution

Testing WCF Services using groovyWS (Apache CXF)

I wanted to use groovyWS to test my WSDL-first .Net WCF web services. By default, WCF doesn't publish its WSDL, it doesn't publish a flat WSDL, and the recommended WsHttpBinding binding makes groovyWS unhappy. Once you work around these things, it is possible to very conveniently write independent dynamic tests for your services.

Since it took a while to collect all this information, I thought I would post my result: a WCF ping service that is interoperable with a groovyWS (i.e. Apache CXF) client.

Cross Platform Java Analysis with DTrace

Since URLs to Jazoon07 slides and blog postings seem to be changing, I'm posting here a stable link to my look into Cross Platform Java Analysis with DTrace.

Update: I gave an updated talk, Dtrace for Java Developers at Jazoon08

Passive Characterization via Statistical Analysis

One area of my research employs the use of statistical analysis to passively characterize an instrument using artifacts inherent in the output it produces. If asked to make an "elevator pitch" for it, I like to refer to similar but easier to describe applications such as the ability to determine an author's gender solely artifacts inherent in the text, or the ability to identify particular anonymous machines on the internet by watching its packets go by using timing crystal artifacts inherent in is message timestamps. I just read a review in the economist for a book called super crunchers, which sounds like it might be another source for such applications.

Week day

In Europe, week numbers are used pretty extensively: The board is due to meet in week 33. Michael will be gone for military service in weeks 24-25. Please submit the quarterly numbers by week 17. Did you know there are at least 6 different ways that weeks are numbered?

Most of Europe seems to follow ISO 8601 numbering, which seems to be what BSD's "ncal -w" computes. But I need to fix my quarterly agenda generator in postscript (that I hacked up starting with pscal code), since I apparently coded up the American numbering by mistake.

If you're always online, there is of course a web-based alternative for looking it up. Its not clear which week number standard yahoo's calendar uses, since I only remember being able to specify which day the week begins on. Maybe it is inferred from your configured time zone?

Programming like it is still 1975?

There is an interesting new project called Varnish, which implements a reverse proxy. It is apparently up to 10 times faster than squid. But more interesting is why.

The software architect, who is usually a kernel programmer, decided to do a "userland program" for a change, as a fun project. Apparently he was disappointed that most programs haven't even taken advantage of things that have already been available in FreeBSD and Linux for at
least 10 years.

Linux threading scalability problems

After a couple of years of work on trying to make the FreeBSD kernel
more concurrent, a couple of kernel hackers started to enable the
concurrency (which is off by default) on an 8 CPU system and started
to see much improved results using a mysql benchmark
to measure it.

To make sure they weren't too far behind linux (which they have been
over the past couple of years) they compared the same benchmark on
the same hardware with the most recent linux and found that linux had
problems scaling when the number of threads was greater than the
number of CPUs.

Some linux hackers began to investigate this and found out that it is
not just mysql. The most likely cause seems to be 2 problems with
glibc - meaning that any program that does threading and uses glibc
(this includes java) would have the problem.

I guess once they identify the problem, it will probably be fixed within
the next few months.