Thursday, October 23, 2008

My Adventures with Ink

I don't know how I've gone this long without attempting to program applications specifically for Tablet PC's before. It was a strange string of events that resulted in my new obsession.

Last Wednesday I went to WIPTE where I presented my poster on STUG and attended several informational sessions. As a result I missed collecting the DyKnow in class problem for the calculus classes. As a result I was given the files on a standard jump drive. This was nothing too crazy and may not seem related to programming, but everything comes together very quickly.

My job in the calculus class can be divided into two main tasks. First, I provide technical support during class to help the students quickly resolve their problems. The other part of my job is managing the in class problem. This job goes even farther than just collecting and managing in that I am designing a system to make easier for future semesters. Next semester when I'm not there they will likely not have a student willing to write a computer program to solve technical problems as they arise.

The accident that started my new obsession was an unintended right click. I simply right clicked one of these DyKnow files and realized that Al Zip could unzip it. I've looked into parsing DyKnow's file format earlier and ran into, what I assumed was, compression. This confirmed my assumption. After unzipping the file it was revealed that DyKnow simply uses XML to store all of the information. A fairly standard approach for storing files that contain digital ink.

As a result I was able to read the file in using a simple C# program and read in the information that I care about. So far I have not done anything too complicated. One of the biggest problem I was having with DyKnow was not having any ability to export the user names of the students whose panels were collected for the in class problem. I spent almost 20 minutes manually transcribing these names. Now I have a simple program that can extract the tames and I can copy and paste them from my simple GUI.

Here is just a short C# code sample that I use to actually open the DyKnow file:

//Open the file
FileStream input = new FileStream(dyKnowFileName, FileMode.Open, FileAccess.Read, FileShare.Read);
//Uncompress the file
GZipStream gzip = new GZipStream(input, CompressionMode.Decompress, true);
//Read the uncompressed file as XML
XmlTextReader textReader = new XmlTextReader(gzip);

This simply opens the file as a stream, uncompresses the stream, then reads the stream as XML. However, this is only the first half of the story.

I ended up having to do some very simple reverse engineering to understand how DyKnow's file format works. I won't post any of that information in this post, but if I post some more code I will explain the tags that I had to use. After I got past the standard compression it was trivial to understand the data that is being stored.

The part that now has me busy is understanding the ink data that is stored as part of the file. I have learned that ink strokes are stored as base64 strings. This appears to be standard for Microsoft inking applications and I simply need to pass it to the correct methods to display it. I have found some example code but I haven't made much progress yet. It seems that ink applications are still slightly complicated to make.


Chad said...

I am having issues with your method of accessing the xml. Did they change the format?

Would you post your source?

(I am attempting to extract the "slide text" from a DyKnow presentation)


Jared said...

I am not sure as I haven't looked into this in a while. I doubt they have changed their format as it would break backward compatibility.

Joel said...

Hey Jared,
I just read your tweet yesterday and was very interested in how you were using the notebook files. I'm a developer at DyKnow, and I think it's great you're finding your own uses for them.

Jared said...

We are currently working at having our paper published at WIPTE 2009. You should be able to read all about it in the conference proceedings this fall.

I work with the calculus teachers to help efficiently analyze panels collected in class to give out grades. I am hoping to continue this work and potentially make my masters thesis about analyzing groups of students work to identify patterns and provide an efficient interface for grading student work using a Tablet PC.

DyKnow provides a very good platform for me to base most of this work on.

Joel said...

Wow, that sounds like a really interesting project (and extremely interesting thesis work), and I look forward to hearing more about it from WIPTE. Have you looked at the Ink recognition tools in .NET? It's been one of those areas that I've been meaning to look at myself. Anyway, if you run into any questions you have, feel free to shoot me an email at jdart (at) dyknow (dot) com (oh how I miss the days when you could just post your email on a website!)

Best of luck,