4 December 20034 December 2003 Upcoming release. 17 November 200317 November 2003 I've applied a slew of patches from Andrew Ruthven. Also, this is an interesting and useful page with information on dashboard. 27 July 200327 July 2003 <edd> given a lat.long clue i can map it to city,state,country

Another brilliant update from Edd. 26 July 200326 July 2003 Gave a talk on Dashboard yesterday at OLS. Slides are here (HTML) [PPT] [OOo].

Some interesting links:

. . .

Hacking

Jimmy, Alex and I pulled an all-night hacking session Thursday night in the hotel room in Ottawa.

Got a first pass of an Evolution mail backend working:

I got graphing back into the codebase, which works pretty well. Because we now send the entire cluepacket to the backends, we had to ask the backends to tag their new clues with the names of the clues they used to generate them. And the same with matches. In addition to helping with the graph, this is important for clustering in the GUI, and probably also for caching.

Jimmy got the indexer working pretty well. He wrote a specialized backend that takes cluepackets from Epiphany and indexes them. So now if you run dashboard and use epiphany as your web browser, every web page you visit gets indexed. The index is also searched as you browse the web, so you get to see web pages that are related to whatever web page you happen to be looking at. This is intensely cool.
22 July 200322 July 2003 The house continues to find itself being rocked.

I've been really busy with work lately, and only found a little time in the last few days to play with Dashboard. Luckily, everyone else is picking up the slack. A couple quick notes about what's going on, since I'm once again short on time.

Edd Dumbill did a big chunk of work today, which is well documented in his recent weblog entry. It's critically lucky that we've got someone who really knows web standards working on this project.

Today Alex Graveley figured out what seems to be a general solution to all the frontend focus issues, making frontend patching/plugin-ing substantially easier. This has been a frustrating issue for me, so I'm really looking forward to seeing his implementation.

Kyle Davis sent in a patch to make Dashboard open your document matches with the appropriate application for their MIME type when you click on them.

Miguel and others on the Mono team fixed all of our Mono bugs, and some we hadn't discovered ourselves that were affecting dashboard. We are a bit more stable now.

A couple interesting things to track:

  • Amazon possibly allowing free text search of books.
  • Possible freedesktop.org standard for bookmarks/url history.
  • And of course there's always more to say and not enough time to say it. Stay tuned for more fun later. 20 July 200320 July 2003 Sun Jul 20 18:14:15 2003

    We now have a mailing list. This is only for people actively contributing to the codebase. The archives are public, though, so anyone can watch our progress.

    . . .

    Sun Jul 20 20:40:29 2003

    You have to be running CVS mono for dashboard to work for you right now. 19 July 200319 July 2003 Sat Jul 19 09:54:11 2003

    Hype

    I'm not going to try to summarize all the events of the last week or so all at once.

    Dashboard's gotten so much attention lately that the hype is eclipsing the code by at least an order of magnitude. A lot of the recent buzz is the result of the demo I did during OSCON, and Tim O'Reilly's subsequent post. But basically we're getting linked from all over.

    So now is the time to redouble our focus on getting the application to actually be usable. :-)

    . . .

    Backend Architecture Thrash

    After a few passes last night, I finally settled on an internal architecture that I know is wrong. I ended up committing it anyway, since it does move the backend interface closer to what we think is correct. Backends can now get the entire cluepacket at once, with multiple ProcessCluePacket invocations as it's chained, so that cross-clue matching can be done.

    What's wrong is that the chaining structure now gets thrown away as the cluepacket gets rewritten. So you can't tell where clues come from, and therefore you can't tell where matches come from. This is a problem because we probably need the ability to sort matches based on their structural origin in the GUI, not just based on their data type.

    For example, if my original frontend cluepacket contains identify information about two people (say, two email addresses), which triggers all kinds of matches about those people, I probably want the UI to segregate the matches which apply to each person.

    The other bad thing about the new architecture is that we lost the ability to draw graphs.

    I'll probably take a cut at fixing this over the weekend.

    . . .

    New Backends

    The good news is that cross-clue matching allows things like the Amazon.com album cover backend, from Thomas Reynolds, to work. Observe:

    (You need cross-clue matching for album covers since the song name and the artist come in separate clues.)

    And on a related note, DJ Adams put together an Amazon book backend:

    It only matches on ISBN numbers right now, but he's adding author matching as well, which will be great.

    Martijn van Beers hacked the bookmarks backends to composite the favicon into the little icon. Here is his first pass:

    Totally slick. This kind of touch is a very important user aid, since it makes it easier to instantly recognize matches.

    . . .

    Status Page

    We now have a frontend/backend status page for tracking all that work. It's incomplete still; I'll try to finish filling it out today. 14 July 200314 July 2003 Massively busy right now. Please hold; I'll be back in full effect on the dashboard in a day or two. Sorry for the productivity gap!

    . . .

    To keep you sated in the meantime, Jimmy recently committed an awesome description of his indexer design. A must read. I cannot wait to get that bad boy hooked up to an IPC mechanism so every application on your desktop can start indexing its objects trivially.

    Also I heard a rumor someone wrote an Amazon CD cover art backend, but I haven't seen it yet.

    . . .

    Please stay tuned for more exciting developments! 12 July 200312 July 2003 Looks like the logjam guys have started on a dashboard frontend patch. 11 July 200311 July 2003 Fri Jul 11 11:35:54 2003

    At OSCON now, just did a demo this morning during Miguel's keynote which went fairly well. A couple people gave a running commentary on IRC. 10 July 200310 July 2003 Thu Jul 10 03:43:39 2003

    Lots of progress on all fronts today. Almost too much to keep track of!

    [ Scroll down a bit if you just want to see new screenshots. ]

    . . .

    Indexer

    Jimmy's indexer getting hooked up was a big step. I don't know much about indexers, but his seems to be pretty generally good and useful.

    Here's roughly how he explained it to me (mistakes my own): the dashboard indexer is a word-stemming vector-based indexer. "Stemming" means that it takes words like "kicking" and removes the stem ("-ing") when it does indexing. After removing common words and stemming, it creates a vector out of all the keywords, where each dimension of the vector represents a keyword, and the magnitude along that axis represents the frequency of the keyword in the indexed text. This vector is the indexed version of the document, and it is stored in the database.

    When you run a query, the query textblock is vectorized, and then the indexer takes the dot product of that vector against the corresponding vector for every document in the index. Whichever one yields the greatest result is the highest-quality match.

    Jimmy currently decays the result quality based on how far into the past the file's mtime is; I'm sure that part of the algorithm will grow in sophistication (using things like the recent files standard to tweak match quality).

    The indexer has a pluggable filter system, and so I think Jimmy may work on HTML indexing next, which will be good since we already have Epiphany sending the full content of all the web pages you visit to the dashboard as you visit them. Having all that content indexed will be pretty nice: your own personal Google.

    I'm really excited about this part of the project. It felt good to watch libremem get removed from CVS today.

    . . .

    Backend Architecture

    Joe is working on fixing up the backend architecture a bit so that match gathering and match rendering are separated. This will be really good since right now if two clues generate matches of the same type, you end up with two totally separate runs through the HTML renderer, with no ability to create any kind of unified UI.

    . . .

    Contributions

    A lot of people have begun contributing fixes and patches, which are sorely needed. There's a lot of progress you can make in the dashboard in a very short time; it's easy to find bugs, so I encourage people to try it out and fix the issues they run into :-).

    Also, most of the mini-project suggestions I made yesterday are still available if anyone wants to pick them up.

    . . .

    My Crap

    Tonight, I worked on three main things.

    . . .

    Textblock Chainer Backend

    I put together a little backend that parses interesting data like URLs and email addresses out of textblock clues and then chains into new clues for other backends to match against. It never actually generates any matches itself: it just does chaining. This way, only one backend needs to know how to parse relevant-looking things out of a textblock.

    I'm not sure this backend makes sense long term -- it just uses regexps and I think Jimmy's vectorizer may be a better place to build from -- but it's a decent stopgap for now.

    . . .

    X-Chat Frontend

    I hacked up an X-Chat frontend plugin. It seems to work pretty well, even handling focus okay (with one or two bugs). X-Chat actually already exposes a 'context' notion to the plugins, so this made things a lot easier. I don't do conversation chunking like the Gaim plugin, and I only send conversation text in the cluepacket, no nicks or channel names or anything like that.

    . . .

    Bugzilla Backend

    Simple bugzilla backend that, given a bug number, looks the bug up on a bugzilla server, using the XML interface and renders a little clickable summary for it.

    . . .

    Putting it all together...

    So, here's my obligatory late-night screenshot of all of this in action simultaneously.

    Pretty nifty. Obviously we're gonna have to start developing some better clue relevance/match quality functions to stop the dashboard from being such a cluttered mess. And we'll need a configuration system of some kind so people can enable the backends they like and disable the ones they don't like. I plan to start focusing heavily on UI in a couple weeks.

    The example above also happens to produce a totally insane cluegraph:

    (There are no RSS matches in that graph because I hit 'graph' before the RSS backend finished downloading and parsing; for some reason it's extremely slow, even if you put the RSS on localhost.)

    . . .

    Stability

    I think my next step is to make this thing stop crashing so much. It's really not that much fun to use when it's always hanging or dying on an uncaught exception or otherwise sucking.

    . . .

    OSCON

    I'm going to be leaving tomorrow afternoon for Portland for the O'Reilly Open Source conference. I've been planning to demo Dashboard for a couple minutes during Miguel's talk Friday morning; hopefully it will work :-). 9 July 20039 July 2003 Wed Jul 9 03:02:26 2003

    Graphing

    I had to spend a lot of time Tuesday doing non-hacking Ximian stuff, so things were a little slow on the dashboard front. But I did take the time to fix the graphing code so that it draws the graph correctly; you may have noticed that there was a missing match in the previous graph. The corrected one is below.

    Also, I added some color. Jimmy calls this "the Ted Turner version." :-)

    . . .

    Frontends

    James Willcox fixed up all the focus issues in the Epiphany patch; bless his heart. I couldn't crack that one.

    Juri Pakaste wrote a frontend for Straw that's now in Straw CVS HEAD. The first thing I want to say about this is that Straw is a pretty sweet RSS aggregator, and I've been looking for one for a while. Though it is a bit hard to get it working, due to all the dependencies.

    The second thing is, it's a pretty neat-o dashboard frontend.

    . . .

    Backends

    Alex has been working on an interesting backend we're calling "the associatrix." Basically, this backend works with insufficient data to help narrow down a match. I won't give it all away now: watch this space.

    Want to help?

    If the dashboard sounds like a cool project to you and you want to dive in and help out, I wrote up a quick list of relatively small tasks that don't require deep knowledge of the dashboard architecture. Here it is.

    • Emacs frontend for hackers.

      The elisp bindings are now done and working, and updated to use the latest cluepacket format.

      It shouldn't be too hard to cut-n-paste some existing elisp code to make an Emacs frontend for developers. Even if the frontend just sent the identifier at the cursor, and maybe the name of the function the cursor is in, that would be sufficient. We would want to add new cluetypes for this, like identifier_c, identifier_java, etc.

      To go with this, there are a bunch of different backends that ought to be built; see below.

    • Man page backend.

      A man page backend could look up C identifiers in sections 2 and 3 of the man pages, and display links to the man pages. If you wanted to be really fly about it, you could even pull out the function prototype and display it in the dashboard.

    • Javadoc, gtkdoc, monodoc backends.

      These would be a little more complicated than the man page backend, but you could parse existing programming documentation -- maybe even docs on the web -- to provide the user with short match data in the dashboard and a link to further information. Header files could be a backend data source as well.

      For extra credit, you might even parse example code out of the docs for a particular class or function and put a little [Example] link at the bottom of the match. Clicking on the link pops up a window with the example code in it, or saves it in a temporary buffer and makes Emacs open it.

    • Epiphany bookmarks/history backends.

      Right now, strangely, we have a Galeon bookmarks backend and an Epiphany frontend. The madness must end.

      A quick Epiphany bookmarks backend should be pretty easy. Epiphany stores its bookmarks in ~/.gnome2/epiphany/bookmarks.rdf. With C#, parsing this XML is trivial.

      Right now I'd try to do URL matches against the bookmark link and keyword matches against the bookmark title; later we can add indexer support for this. You can even probably steal a lot of the code from the existing Galeon backend.

      An Epiphany URL history backend should be equally easy; the corresponding file is ~/.gnome2/epiphany/ephy-history.xml.

    • id3 tag backend.

      It shouldn't be hard to find the user's mp3s and parse out their id3 tags. The tricky bit might be matching against incoming cluepackets. For now I'd only look at textblock/htmlblock cluepackets, and do a substring search on song/artist titles.

      Clicking on music matches should, of course, enqueue them in your mp3 player.

    • Gthumb backend.

      Gthumb allows you to annotate your photos with keywords and notes. We already have a Gthumb frontend; it'd be great if the dashboard could match against these.

      You could also search on date clues, and show the most-relevant photo from that particular day. I'm envisioning reading an email from three weeks ago, and glancing at the dashboard and saying "Oh! June 25th! That was that sunny day I took the picture of the cow." Or whatever. That might be pretty specific to me ;-).

    • Nautilus custom icon support.

      Nautilus allows you to specify custom icons for files and directories. When the dashboard displays a match for a file, it should use Nautilus's custom icon.

      This shouldn't be as hard as it sounds; the custom icons are stored in ~/.nautilus/metafiles in a bunch of XML files. It's probably only a few lines of code to parse these, extract the custom icon path, and stuff into into the matches returned by the current documents backend.

    • Drag and drop.

      Clicking on matches and loading them is nice, but sometimes you want to send them to another application, or copy them somewhere. GtkHTML, the HTML-display widget we are using, does actually support drag operations; it emits a signal when a drag operation starts, and you give it the drag data.

      In theory this shouldn't be too hard to implement; if no one else does it, I'm certainly going to, since in my mind the ultimate demo is:

      1. In an IM conversation with someone, they ask you about some project you've been working on.
      2. The dashboard notices what you're talking about and matches your latest design document for the project.
      3. You say to your friend: "Check out my current design." and drag and drop the document from the dashboard onto your IM window.
      4. Gaim transfers the file to the other person.

      This one might take a little research and trial-and-error (and would require a Gaim patch to accept file drags) but it's for sure worth it.

    • Frontend/dashboard packages.

      Right now the dashboard is pretty hard for people to test, largely because it requires applying a bunch of patches to frontend applications like Epiphany and Evolution. It would be massively helpful if someone took the time to build RPMs or Debian packages of these custom-patched apps, and of the dashboard itself.

      It's probably a pretty big conceit to call this task "easy" :-).

    Of course, there are tons of other tasks, so this is pretty incomplete, but hopefully it's a good good starter list for now. I'll put a more complete one together at some point in the future.

    . . .

    Wed Jul 9 10:59:39 2003

    Someone on IRC just suggested the idea of a CVS backend, with recent commits from whomever you're interacting with, including links to Bonsai, etc. Great idea!

    . . .

    Wed Jul 9 12:14:38 2003

    A really serious moment of truth: Jimmy's indexer is hooked up and is working as a backend.

    Wow. This is pretty damn cool :-).

    . . .

    Here's a new graph for ya.

    . . .

    Wed Jul 9 17:21:19 2003

    Jonas Heylen just wrote, and I just committed for him, an Epiphany Bookmarks backend. Man, that was fast! Scratch one off the TODO list from this morning.

    In other news, Mark Humphreys seems to have a RhythmBox frontend up and running. And better still, he's working on a backend now. I can't wait to see it.

    Here's an idea for someone: build a bugzilla backend.

    . . .

    Wed Jul 9 17:21:19 2003

    A couple predictions, so I look smart in a few months:

    • I predict that every single part of our current design is wrong.
    • I predict that many people will continue to confuse the idea of an association engine with the idea of a notification system. Without any existing notification system to refer to, dashboard looks too much like a message bus for people not to make this mistake.
    • I predict that in the next three or four months we will see several people start writing their own dashboards. This is just too sweet a spot for other people to leave to one implementation project.

    • I predict that we will need to build several custom dashboard user interfaces for special purposes. Hopefully they will be able to share a common engine, but this might not end up being the right thing.
    • I predict that Dashboard-like projects will create completely new, hitherto-unseen user interface patterns.
    Okay, that's enough self-congratulation for today. Stay tuned for some cool screenshots that expand your idea of what the dashboard can do... 8 July 20038 July 2003 Tue Jul 8 03:47:15 2003

    I got the new architecture committed, with cluechaining working. There's still some cruft, but it's substantially cleaner, and the match and cluechain caches are in-place and working beautifully. There's no expiry on the caches yet; we'll have to fix that.

    I think we can stick with this engine architecture for a little while -- hopefully a week or more ;-).

    Now that the association engine is a little more solid, we can probably think a bit about how to allow people to write non-C# backends. As well as doing a better job of match sorting.

    . . .

    I also added a neat debugging hack that shows you the cluechain graph if you want to know how the dashboard reached its matches, and how the cluechains were built. There's a "Graph" button in the upper-right corner of the dashboard which creates the graph and launches eog.

    Ellipses are clues, boxes are matches, solid lines indicate a clue generating a match, and dotted lines indicate cluechaining.

    It only works if you have graphviz installed. 7 July 20037 July 2003 Mon Jul 7 15:47:01 2003

    Edd Dumbill's GNOME Phone Manager tool now sends cluepackets to the dashboard when you receive an SMS text message on your bluetooth mobile phone.

    <CluePacket>
         <Frontend>Phone Manager</Frontend>
         <Context>Text message</Context>
         <Focused>true</Focused>
         <Clue Type="phone" Relevance="10">phone number of sender</Clue>
         <Clue Type="textblock" Relevance="10">Text message.</Clue>
    </CluePacket>

    . . .

    I hacked the Gaimlog backend to put the user's buddy icon in.

    Andrew Sobala wrote a Gthumb frontend patch.

    . . .

    Mon Jul 7 22:50:40 2003

    Oh shit, Edd's FOAFBot backend is amazing. He posted a screenshot on his blog of the Phone Manager frontend triggering a cluechaining lookup into his bookmarks, which I've replicated below. Fucking awesome.

    . . .

    By the way, we're not using the term "cluepacket rewriting" anymore. As it turns out, we don't do any rewriting: we only generate additional clues from other clues, starting at the original cluepacket. These "chain" all the way down till you get your full match list. So the term "cluechaining" is now in vogue. Stay tuned for a new cluechaining-based architecture and some sweet screenshots... 6 July 20036 July 2003 Sun Jul 6 05:58:37 2003

    I think the screenshot below speaks for itself.

    Credit must be given to Jimmy, who wrote the cluepacket rewriting code in the addressbook, and who first thought up the whole clue-augmentation process.

    My new RSS backend was a really simple addition, that could only have been done once cluepacket rewriting actually worked.

    I tried for about two hours and could not crack the focus issue in the Epiphany frontend patch. Hopefully someone can help me. I think the Evolution focus handling is right, though it is sending two cluepackets when the window regains focus for some reason.

    5 July 20035 July 2003 Sat Jul 5 08:37:00 2003

    Pretty fucking decent progress today so far.

    Indexer

    The big news today was that Jimmy checked in the beginnings of his C#-based stemming indexer.

    This should be massively helpful; the idea is to expose it to the dashboard and to the backends as a utility class which they can use for matching and indexing.

    We're also going to expose a simple indexing interface to the outside world, similar to the current cluepacket interface, so that existing applications can index their objects really easily. For example, your web browser is going to index all of the web pages you visit, so we'll have really good matching on your recently-visited web pages.

    Dashboard backends for data stores that already have their own indexes, like Evolution mail, obviously won't use this indexer.

    I'm really excited about the possibilities: not only can we index your documents, but we can index the web pages you visit, your IM and IRC logs, and any other text you interact with, incrementally and in real-time.

    . . .

    Addressbook

    I finally have all the bugs worked out in the addressbook/IM changes I started yesterday: I can query the addressbook from C# and actually get good results and the IM fields work properly in the UI everywhere.

    I also went ahead and wrote a simple addressbook backend for the dashboard. It's pretty dope, especially since it all updates in real-time from the Evolution addressbook.

    The pain of working inside the addressbook's current architecture is pretty high, however, so I opted not to do the multiple-accounts-per-service stuff I was describing yesterday; someone else can add that if they want.

    . . .

    Frontends

    The big news in frontend-land is that I changed the cluepacket spec; you can read about the format here.

    I don't think this will have to change again for a while, except for adding new clue types.

    Sean Egan put a patch into Gaim CVS to add a plugin event for when the user manually switches conversations. This ended up not being nearly sufficient to solve my focus issues, but I eventually came up with a pretty simple solution. The Gaim plugin now has perfect focus handling, and otherwise implements all the frontend changes the new cluepacket formats requires. I now consider this the reference frontend implementation.

    The Evolution mail and Epiphany patches still need to be updated to reflect the format changes, and to handle focus. It would probably only take a few hours for each of these, if anyone wants to volunteer. :-)

    On the other hand, Peter already updated all of the Perl and Python bindings, and he wrote a really nice test server in Python for people who want to debug their frontends without actually running the dashboard.

    Peter's elisp frontend is now in CVS as well.

    . . .

    "Cluepacket Rewriting"

    In case you're wondering why I'm spending so much time on the addressbook, it's because we think the addressbook might be the most important association index on the computer.

    As an example, if I get an IM from "SeanEgn", then I'd like the dashboard to show me the last couple of emails I got from Sean.

    Unfortunately, Sean's email address is "sean.egan@foo.edu", not "SeanEgn@*". So, since a simple text search won't work, that means that somehow the dashboard has to draw an association between "SeanEgn" and "sean.egan@foo.edu" in order for the mail backend to find mails that correspond to the person who just IM'd me.

    Our idea for dealing with this is through a process we are snootily calling "cluepacket rewriting." Hopefully, it will work like this:

    1. SeanEgn IMs me; the IM client sends the following cluepacket to the dashboard:

      <CluePacket>
            ...
            <Clue Type="aim_name" Relevance="10">SeanEgn</Clue>
            ...
      </CluePacket>


    2. The dashboard sends this cluepacket out to all the backends for match finding.

    3. The addressbook backend, like all the other backends, receives the cluepacket, and searches its indexes (the user's addressbooks) for matches.

    4. The addressbook backend finds an address card that matches, and sends it to the dashboard's match aggregator to be prioritized and (if it's good enough) displayed to the user:

      <card>
            <fname>Sean Egan</fname>
            <email>sean.egan@foo.edu</email>
            ...
      </card>


    5. But the addressbook backend takes another step. It knows that some of the properties associated with its match are useful associative data, so it rewrites the cluepacket to include that information:

      <CluePacket>
            ...
            <Clue Type="aim_name" Relevance="10">SeanEgn</Clue>
            <Clue Type="email" Relevance="10">sean.egan@foo.edu</Clue>
            ...
      </CluePacket>


    6. The dashboard then resends this rewritten cluepacket to all the backends for matching again.

    7. The process can then repeat; we might have to put a hops-to-live kind of limit on it.
    Of course we won't have any idea if this makes any sense till we try it. One of the things about the dashboard is that the architecture kind of has to be built from the outside in; it's really easy to make design mistakes if you don't have actual data (cluepackets and indexes) to work with. For example, I never would have guessed focus would have been such a problem if I hadn't started by writing a few frontends and playing with it.

    . . .

    Helping

    There's a huge amount to do; if anyone wants to help, join us on #dashboard on gimpnet, or mail me.

    . . .

    Sat Jul 5 21:17:35 2003

    Imran Patel is writing an XMMS frontend plugin. Yay! This is slightly tricky, since it probably requires that we implement additive cluepackets...

    . . .

    If anyone wants a fun project, an RSS backend which displays the last couple blog entries from whomever you're IMing or reading email from would be a great little project, once we have cluepacket rewriting in place (tonight?).

    . . .

    Sat Jul 5 23:05:31 2003

    Alex Graveley just committed an amazing frontend that uses the accessibility engine to send the text that you're typing to the dashboard as cluepacket whenever you finish a word or sentence. Fucking awesome.

    This at least doubles the need for an "additive" cluepacket property.

    . . .

    Sun Jul 6 01:26:46 2003

    Joe just committed a better backend architecture, with semi-working additive properties.

    I've updated the Evolution mail frontend to use the new cluepacket format and to handle focus. Working on Epiphany now, then I'll commit. 4 July 20034 July 2003 Phew, long night.

    I wrote a really simple synchronous interface to the Evolution addressbook, made it serialize address cards to XML, added RSS and IM fields to the addressbook backend, and built a UI for it. (Of course, you can always serialize to vCard, but XML is nice to have because everyone can parse it.)

    The vCard exporting is compatible with the way it's done in OS X, so you can import Evolution cards into the OS X addressbook and vice versa, and the IM fields are preserved.
    Then, I hooked the Evo addressbook search interface up to the dashboard, so the dashboard can now search your Evolution addressbook, whether or not Evolution is running, using Evolution's own search infrastructure. Bam!

    Unfortunately, my UI sucks and no part of my implementation allows you to have multiple IM accounts under the same service. Oops.

    So when I wake up, next steps are:
    • Refactor the Evolution addressbook patch so that you can have multiple IM accounts for each service
    • Rewrite the notebook page to use a treeview with dropdowns
    • Document the new dashboard backend interface so other people can write backends
    • Get all this shit into CVS
    One thing that someone could do that would be superfly would be to write a Gaim plugin that integrates with the Evolution addressbook. It would basically write new cards into the addressbook for new buddies, import cards which contain IM names, and give you a nice interface for associating a buddy in your buddylist with an address card in your Evolution addressbook. I think it's about two days of work.

    Also, for the focus issue, I pretty much have a solution, though it will require updating the cluepacket format and doing some substantial hackery in all the frontends.

    The real issue that's starting to loom high on the horizon is that the dashboard is still totally useless :-).

    Luckily, it's the weekend. Time is on our side.

    . . .

    Okay, maybe I'm the last person to hear about this, but I just learned that there's a group of people out there building a panel-like sidebar for Windows called ... the dashboard. (Notice: they cannot spell.)

    I'm kinda pissed, because I thought "dashboard" was a reasonably clever name, and now everyone's going to think I copied it from these losers.

    . . .

    Josh's Java frontend binding is now in CVS.

    . . .

    Finally got the new cluepacket interface documented.

    Did some source tree rearranging, too. It's getting cleaner; Joe has done a great job of splitting the main dashboard code into the appropriate classes.

    . . .

    We are now hanging out in #dashboard on gimpnet. 3 July 20033 July 2003 Got drunk last night, not much progress.

    Jimmy and Alex and I met up for coffee, though, and we worked out most of the backend architecture. I'm going to redo the internals to our new design as soon as I catch up on some other things in the frontends.

    One of the things about this project is that involves lots of things dorks like to pontificate about:

    • Heuristics
    • Architectures involving message passing
    • User interfaces
    • Plugins
    • Plugins that do message passing
    • Plugins that provide user interface elements
    • Google
    • Plugins that talk to Google
    • Programming languages: why some suck and others rule
    ...and so on. The consequence of this is that you get lots of people giving you "input" that you really could do without.

    I guess what I'm saying is, if I hear from one more person about the semantic web, and how we really ought to look at that, blood will run freely.

    Maybe I'm just getting ornery with age.

    . . .

    My current TODO list:

    • Some improvements for the Evolution mail frontend
    • Evolution calendar frontend
    • Galeon frontend
    • Poke Jimmy with a sharp stick till he writes a new indexer or finds one that doesn't suck
    • URL history backend
    • Try to solve the focus problem
    Some things I'm not going to do soon that someone else could do if they had the time and inclination:
    • Emacs frontend
    • X-Chat frontend plugin
    • Gthumb frontend plugin
    • Gthumb backend
    • OpenOffice frontend plugins
    • Think about how libgsf can help us
    Most of the easy-for-someone-else-to-do work is in the frontend plugins, but unfortunately I'm pretty much doing most of these already. I really want to get the core plugins done and behind me. The backends will be really easy to write as soon as Jimmy and I hammer out the interfaces a bit more.

    . . .

    In other news, Chris Lahey is already using Peter's Python bindings to write a frontend plugin for his IRC client (which I don't have a link for).

    . . .

    Waitaminute. Why am I writing Galeon stuff? Awe shit, time to redo everything for Epiphany.

    Which is now underway...

    . . .

    A note about expectations: dashboard is not yet useful in any way. Don't get your hopes up if you're thinking about actually using it.

    . . .

    Thanks to Josh Lucas, We now have a Java frontend class. Woop!

    Gotta get this backend architecture in place so we can get the backend stuff moving as quickly...

    . . .

    Working on the addressbook backend, using that as a driver to get the right backend architecture in place in the dashboard in general. It's dope, I'm actually using Evolution's addressbook searching functions.

    Meanwhile, Jimmy is writing a new indexer -- thank god -- and Joe is writing a new alternative frontend "library" that uses GIOChannels and therefore integrates with the mainloop of Gtk+ apps, etc.

    . . .

    One important thing to realize is that it would never be possible to write something like the dashboard in a world where you can't get the source code to your applications. This is the whole "basis for innovation" thing we're always talking about.

    . . .

    James Willcox just committed an Epiphany frontend patch.

    . . .

    So, in September of 1998, I wrote a little note about storing URLs in a searchable repository. And just a minute ago, when I remembered this, it struck me that my idea back then presaged dashboard pretty well.

    Maybe we're all just born with one or two ideas and we spend the rest of our lives articulating them in different ways. ;-)

    . . .

    Peter is working on elisp bindings so you can use Emacs as a dashboard frontend.

    . . .

    Josh Lucas is back again, this time with an X-Chat frontend plugin underway... 2 July 20032 July 2003 Put together a first pass at an Evolution mail frontend patch. It builds a cluepacket out of the sender's name, email address, and the message subject and body.

    Unfortunately, it appears to be triggering a bug in the indexing backend which causes the dashboard to freeze. That's why I disabled savant for the screenshot above. I'm not sure if the issue is in libremem or in the C# wrappers, or in Mono. It might be happening on really large queries, or something.

    . . .

    Peter Teichman is writing Python and Perl modules for sending cluepackets to the dashboard. Sweet! That'll make it trivial to turn python and perl apps or python/perl-scriptable apps into dashboard frontends.

    One of the next things we have to do is to come up with a backend architecture that's as simple as our frontend architecture, so writing backends is really easy.

    . . .

    Cleaned up the Evolution Mailer patch a lot. The patch is so simple, it is really starting to reflect the simplicity I want in the frontend support code.

    Some new utility functions for frontends helped with this a lot. Also cleaned up the Gaim plugin to match.

    Peter's Perl and Python frontend bindings are now in CVS. He also wrote a cluepacket DTD, which I am certain will be out of date within 48 hours. 1 July 20031 July 2003 I want to build an Xr-based UI for Dashboard.

    Spent some time Sunday night looking at the GNOME accessibility framework to see if I could trawl all the data I want out of frontend applications with it, instead of writing per-application patches and plugins. Unfortunately it's woefully underimplemented and therefore essentially not usable.

    Thoughts on dashboard's relation to semantic web, RDF. I think this project has a lot of dorky-blogger-meme-virulence potential.

    Up till 2am last night working on the Evolution frontend. Sadly I have real work to do today, but hopefully I can get it done tonight. 30 June 200330 June 2003 Added a patch for Gaim so that it sends cluepackets to the dashboard as you talk to people. TODO on that patch:

    • Set relevances on all the clues.
    • Send the last 20 lines of chat in chunks instead of all at once, with decreasing relevance as the chunks get older.
    • Make the connect() nonblocking; this didn't work for me for some reason.
    • Strip the timestamps and buddy names from the chat.
    • Send a new cluepacket whenever you switch tabs/focus a new IM window.
    There's also a standalone tool to send cluepackets.

    Current screenshot:

    We're going to need duplicate match pruning soon.

    . . .

    We've uncovered a few bugs in Mono, too. Jimmy's Google backend tripped a chunking bug in the WebResponse class, and while wrapping libremem, we found out that we have to do some nasty manual marshaling.

    . . .

    Sean Egan tells me that he's going to add plugin events for switching tabs and focusing conversations in Gaim, which will mean my whole patch can become a plugin. That'll be good.

    . . .

    Okay, the Gaim frontend support is now a Gaim plugin. Sean, I need new plugin events for whenever a given conversation gets focused.

    Which makes me think: we need a way for a frontend to send a cluepacket which just references a previous cluepacket. So when you switch focus around windows, the dashboard instantly updates to cached matchsets, instead of regenerating all the matches.

    We also need double-buffering in the UI.

    My next steps, I think:

    1. Refine Gaim plugin.
    2. Write Evolution frontend.
    3. Redo UI. I want to compartmentalize the different match types into different parts of screen space.
    Another thought on that cluepacket-by-reference idea: maybe the dashboard just needs a general way to figure out what frontends actually have focus. Then they could all send cluepackets all the time, and if the dashboard has enough resources, it could precache matches, and only switch the visible matchset when a given frontend gets focused.

    . . .

    The Gaim plugin is basically finished; it's non-blocking, does the chunking, strips all the timestamps, etc. I have this focus problem still: unfocused, even iconified conversation windows transmit cluepackets (bad), and when you focus a conversation, you really want it to resend its cluepacket (good). This is going to come up again and again; there might be a general solution involing metacity interaction.

    Also, I added a quick TODO of small tasks for anyone who wants to join the fun.

    . . .

    Totally shocked and pleased to see that the illustrious Matthew Garrett (aka "The Dasher Guy") has managed to get Dashboard running. I think Matthew's the first person I don't live near who's made the thing run at all :-).

    . . .

    We have to ditch this crappy indexer. Why is it matching files with a 2-year-old mtime? Why? 29 June 200329 June 2003 Initial code is now in GNOME CVS. Please note the README.

    Jimmy and I are working on hooking up the "word-stemming" Savant indexer from the Emacs-based remembrance agent. The code for Savant is disguting; it's meant to be an interim solution until we either find a better indexer or write one.

    Right now, we're sort of trying to get the dashboard into a demoable state for OSCON, where it may be shown during Miguel's keynote, depending on whether or not it works :-). 9 March 20039 March 2003 One of the things I've been hacking on with a high level of intermittence is my dashboard idea. The basic notion is automatic search of your personal information space as you go about your regular daily activities: reading and writing email, browsing the web, talking to people on IM (at least, these are my regular daily activities).

    As you interact with a given object (an email, a web page, a person on IM), the dashboard is supposed to automatically populate with nice, visual representations of related objects in your little universe. And then, of course, there are various ways that you can do things with those objects: click to open or activate them, drag and drop them onto another application, etc.

    I ended up writing most of the code in Mono, but I have a really tiny, lightweight C library that is used to connect the "frontend" applications (the ones you're interacting with most of your day) to the central association engine.

    A few parameters are guiding the design and implementation:

    • Integration with frontend applications must be trivial. Ideally, one line of code per frontend. The frontend library depends only on glibc.
    • Zero user interaction should be required in almost all cases. I'm uninterested in having users manually train or configure this system. All the relevance, scoring and sorting algorithms are simple, componetized and self-tuning. This principle also means that all operations should be one-click; i.e. expanding subtrees in the match list is unacceptable.
    • Authorship of new backends (the things that find, index, and match relevant objects with whatever the user is doing) must be straightforward and compartmentalized. Right now, you only have to implement two functions.
    • Highly visual related-object presentation. The idea here is that the dashboard is never something you stare at; it is something you glance at. This means foregoing text for images whenever possible, but it also means not having to scroll (i.e., the scoring/sorting has to be good).
    • Work well despite the fact that Linux desktop lack most of the infrastructure I need.
    This is a very stimulating project precisely because search is an extremely rich and deep problem space. I am starting to understand why the guys at Google have so much fun. Every corner I turn, I find myself running into whole reams of interesting thought problems.

    Automatic associated object retrieval just happens to touch on a whole lot of very big problems, many of which are either open research topics or issues which require a lot of unwritten code to address. Just a few examples:

    • Keyword extraction. Many of the frontend objects that matter to people are text-rich, like emails, web pages, and IM conversations, and in order to find useful related objects, you want to pull the keywords out of them.
    • Automatic data/history storage. Computers right now (and Linux computer in particular) regularly discard (or do not make available) a lot of potentially useful data that would be great to mine. Things like: what applications were on the screen when a given document was written, web pages from which you cut and pasted text in composing an email, URLs that people emailed you, text from web pages you visited, the source of files that you downloaded, etc.
    • Structured fields from arbitrary documents. Given any file, who wrote it? Who edited it? What is its title? When was it created? ...
    And these are just a few issues. There are tons of specific problems (no automatic file indexing facilities exist in Linux; we lack a simple and widely-deployed IPC framework, ...) and the really scary general ones (extracting semantics from text).

    This abundance of greenfield is dangerous: there are a large number of papers that address these subjects, but very few useful implementations.

    A lot of people have pointed out the Emacs rememberance agent to me, which is the closest comparable project that I know of.

    But actually, there is one other group of people working on this issue: Microsoft. This is exactly what they want to be able to do with Longhorn. Longhorn is going right after Google, going right after the problem of actually putting "information at your fingertips." So I was kinda trying to beat them to the punch, but it turns out they have more money than I do.

    This is, unfortunately, the kind of project that everyone likes to give you reams of really useless and irritating advice about, so I haven't been talking about it much. I'll be putting more stuff up here as I go, though. Don't expect a lot of regularity.