Now, obviously if things did not turn out well after you made some decision you can only wish for the chance to go back and do it differently. This would be only reasonable, who doesn’t want to improve their situation? The reason I usually answer in the negative is not one of stubbornness in refusing to believe I made a mistake, it is simply a matter of attitude. I believe that lingering on the past and lamenting a decision has far worse consequences on an outcome and your ability to adapt to the situation than (almost) any bad choice (a counterargument can be made to an extreme, but my topics are for basic life decisions, not ones that end in fatality or some other trauma). For me, the most important aspects of evaluating a poor choice are as follows (not novel in any way): maintaining control and understanding alternatives.

The worst thing you can do to turn an already bad situation into a worse one is lose control. I don’t mean in just a physical sense. Any situation can be salvaged (remember.. generalities, not absolutes) to glean something. The reason you can have thoughts like “regrets” is because you are able to evaluate changes in your life relative to one another. The critical thing is analyze and understand your reactions to the changes and note them for future use; learning from your mistakes (nothing fancy here, we have been told this from gradeschool). But in order to do that you have to make mistakes, that part somehow gets conveniently left out. No one is perfect! But your ability to adapt to a situation and proceed forward with an understanding that you still have control is a major factor in being able to deal with the majority of shit life can deal out.

Once you know you haven’t lost control, you have to understand that everything is relative. Good for one is bad for another. You may have a vague idea of “universal” truths, but overall, it is just societal influence that tends to guide our daily lives. Don’t let one person’s idea of “values” interfere with your ability to reason and determine your own values. And certainly don’t be afraid to change your perspective in light of new arguments. Everyone is at a different place in their lives and has their own (skewed) perspective on how they think things should be. Don’t try to make their reality your reality and vice versa.

While the last paragraph may seem a bit off topic, it is far from it. The reason understanding how others see your outcome is important is because we often judge ourselves from the perspective of society. We think about what is “expected” of us and what the norms are. This contributes to the feelings of regret. People often forget that their current situation has altered their understanding of themselves and the world around them. They then make the mistake of analyzing their original choice but in their current context. This is incorrect as the new information may not have been available at the time or understood to mean what it does now. Always analyze the past in the proper context and apply that analysis going forward to improve your situation.

If you life your life worrying about having regrets you will find it increasingly harder and harder to take the bold steps that can make life interesting. Always focus on the simple things: what did I learn about myself, how did I react, and what is my next step from here to where I want to be.

p.s. I don’t know why I wrote this. I have no real decisions to make or regrets Just wanted to share some thoughts; it is a blog isn’t it?

Scraping a normal, up to date website, over SSL is not a problem; however, when you encounter a site like SPSU’s course schedules you realize why web browsers are the magical things they are. At first glance, this looks like a normal site served up over ssl. It even has the green lock in chrome to indicate everything is “secure” and give me a warm fuzzy feeling. However, if I try to use ‘curl’ on this page, it bawks with:

curl https://banweb.spsu.edu/pls/PROD/schedule.main
curl: (35) error:14077417:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert illegal parameter

This was strange to me as my browser loaded the page just fine. Chrome does however give a hint as to something being wrong. When I click on the green lock icon, I get the usual certificate notice as well as an additional piece of information:

The connection had to be retried using SSL 3.0. This typically means that the server is using very old software and may have other security issues.

Clearly something is not right with the webserver here and that is what is causing curl, and subsequently my nodejs requests, to fail.

Doing some basic digging, I was able to find this nice little tidbit in the openssl documentation:

SSL_OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION

Allow legacy insecure renegotiation between OpenSSL and unpatched clients or servers. See the SECURE RENEGOTIATION section for more details.

Now that I had this option to try, I had to figure out a way to make node.js use it. Reading some code led me to understand that the ‘Agent’ is the object which handles the options for http(s) requests which are ultimately passed to the SSL context. When creating the agent, you pass in an options object, this object contains two fields which need to be used for this case: secureProtocol and secureOptions.

secureProtocol specifies which negotiation method to use
secureOptions are the flags passed to the context to change the behavior

My final agent object had the following form:

var agent = new Agent({
secureProtocol: 'SSLv3_method',
secureOptions: require('constants').SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS
});


My https request options object just need to include the ‘agent’ key and have it set to the agent object I created. This filtered down to the underlying ssl context and set the context options. The request was a success!

This will only work with node.js >= 0.6.3 because the constants are only available after that. Other constants from the bug workaround section of the openssl documentation are also available for your needs.

References:
ssl docs: http://www.openssl.org/docs/ssl/SSL_CTX_set_options.html
nodejs docs: http://nodejs.org/docs/v0.6.2/api/http.html#http.Agent

headphones before modification (one pad removed)

My thinking was to remove the wire connecting the two ear pieces. Then see if I could cut down the plastic part supporting the wire. Initially I had planned to also remove the actual driver from the plastic encasing but that proved unnecessary since the side with the plastic was perfect for the slot in the helmet.

after modifications

After removing the plastic support, I plugged the hole with some hot glue and installed into the helmet. Result: awesome!

headphones installed

Georgia Tech - Spring 2012

Full Res svg: http://www.shtylman.com/misc/courseoff/spring_2012.svg

Created using the d3 javascript library: http://mbostock.github.com/d3/ 

Before jumping into my benchmarking details, there are two important concepts to understand behind using MongoDB for message passing.

The tailable cursor gets its inspiration from the “tail -f” command in unix systems. For those unfamiliar with what that does, the idea is to open a file, listen for new additions to the end of the file, and print those. The program does not terminate when it reaches the end of the file, instead, it waits for more data. Using this approach, a simple message queue can be created with MongoDB. A producer creates messages and inserts them into MongoDB and then a consumer(s) is able to read those messages. While this can be simulated by keeping track of document IDs (in MongoDB and other database systems), in MongoDB the tailable cursor is actually supported server side and tracks the last returned document. This is much cheaper than having to constantly requery.

In MongoDB tailable cursors can only be opened on capped collections. This is because a capped collection is a fixed size and only allows insertions. This means that once the number of documents exhausts the collection size, newly written documents will start overwriting the first inserted documents. An important note about capped collections is that insertion order is the natural sort order. This means that when the tail cursor fetches documents, it will get them back in the order they were inserted. If you have more than one producer and insertion order matters, then you will need to either use separate collections or do your own reordering consumer side. For my examples, there is only one producer and thus natural (insertion) order is acceptable.

Now that we have some background on tailable cursors and capped collections, we can cover the benchmark setup. To perform the benchmark, I will have one producer and one consumer. The producer will get create a document of the format (Where the current time will have microsecond precision):

{ time: (current time in seconds) }

The producer will generate 100,000 of these messages and insert them into the collection. To avoid dealing with a slow consumer, the collection will be made big enough to hold all of the possible messages:

db.createCollection("messages", { size: 100000000, capped: true })

The producer is a simple python application:

#!/usr/bin/python

import time
from pymongo import Connection

conn = Connection()
db = conn.queues
coll = db.messages

start = time.time()

count = 100000
for i in range(0, count):
    coll.insert({ "time": time.time()})
    #This generates messages at a rate of about 3,900 msg/s
    time.sleep(0.0001)

end = time.time()
print("total: ", count)
print("msg/s: ", count/(end - start))

The consumer is a c++ client modeled after the default c++ tailable cursor example on the MongoDB page. It gets the system time upon reading a document and computes the latency between insert and read. The consumer is started before the producer:

#include <sys/time.h>

#include <mongo/client/dbclient.h>

using namespace mongo;                                                                                                  

int main(int argc, char* argv[])
{
    DBClientConnection conn;
    conn.connect("localhost");                                                                                          

    // minKey is smaller than any other possible value
    BSONElement lastId = minKey.firstElement();                                                                         

    // { $natural : 1 } means in forward capped collection insertion order
    Query query = Query().sort("$natural");                                                                             

    cout << "loc,val" << std::endl;
    uint32_t i = 0;
    struct timeval tv;
    while( true ) {
        auto_ptr c = conn.query("queues.messages", query, 0, 0, 0,
                QueryOption_CursorTailable);
        while( true ) {
            if( !c->more() ) {
                if( c->isDead() )
                    break;
                sleepsecs(1);
                continue; // we will try more() again
            }

            const BSONObj& o = c->next();
            lastId = o["_id"];
            const double time = o["time"].Double();

            gettimeofday(&tv, NULL);
            const double curr = tv.tv_sec + tv.tv_usec / 1000000.0;
            cout << i++ << ", " << curr - time << endl;
        }

        // prepare to requery from where we left off
        query = QUERY( "_id" << GT << lastId ).sort("$natural");
    }

    return 0;
}

Running the consumer/producer pair generated the following latency graph:

Default c++ consumer

The graph is not lines, but actually 100,000 points that represent the N-th packet versus the latency (read time – write time). The above graph has a standard deviation of: 0.29693s. While these results are not horrible, the sawtooth nature of the graph would indicate that there is something strange going on and that ideally, we can do much better and decrease the latency. I ran a simple consumer (without clearing the producer data) to see if the bottleneck was in the write side or read side:

Because the read only graph is essentially linear, it means that the consumer has no problem in reading all of the data if it is available and likewise, the producer did not have problems inserting the data. (The latency is irrelevant here as the data was inserted before the consumer had to read it). So given the above graph, we know the consumer can handle reading the data consistently if it is available, so the sawtooth pattern must emerge from what happens when new data is being inserted.

Taking another look at the Default C++ consumer graph, I noted that the consumer only goes back to ~1s latency when the previous latency was almost 0. This information, along with the c++ code which contains a sleep for 1 second was the “aha!” moment. When there are no more documents to read, the client waits for new documents to show up. Unfortunately, instead of being woken up when new documents are available, the default example uses a simple poll. This is expensive if our goal is to minimize latency. To test my theory I removed the sleep statement from the code and ran the consumer/producer pair again:

No Sleep

This method produced a standard deviation of 0.000739s. However, this is not the end solution as it has one major problem; it uses a tight loop which causes high CPU load locally and worse yet, thrashes the mongo server. Looking over the c++ driver API I found an interesting query option: AwaitData defined to: “If we are at the end of the data, block for a while rather than returning no data. After a timeout period, we do return as normal.”

The AwaitData option sounded like just the thing I needed. I modified the query code to the following:

auto_ptr c = conn.query("queues.messages", query, 0, 0, 0, QueryOption_CursorTailable | QueryOption_AwaitData);

It is important to note that this causes the “more” cursor method call to block for a short time. If you cannot afford to block, this will not be a solution for you (see conclusion about some thoughts about evented systems)

And sure enough, when there were no more records to read, the local CPU would not spin. Yet I still noticed a problem. Since I start my consumer before the producer, I found that a bunch of re-querying was happening until the first record was inserted. This is due to a “dead” cursor. When the capped collection is empty, MongoDB does not keep a cursor open server side and thus the whole “tail” process never starts until at least one record is available. Given that I did not want to spin the local CPU or thrash the server, I inserted a sleep before the “break” when the cursor was “dead”. This would cause a requery every 1 second until a cursor could be created (at least 1 record was inserted). While this did solve the spinning CPU problem, I found this solution caused a high initial latencies because of the sleep before the query:

Sleep before requery

Once the initial query happened, and a valid cursor obtained, then further queries were not needed and the latency remained very low. However the start up latencies were poor because we could be anywhere in the 1 second sleep before we are able to re-query.

To solve this problem, I used a dummy document to prime the collection and cursor. Given that even with a single document, a cursor would be created server side and the client would no longer have to query waiting for a valid cursor. While this solution is also not optimal, it does avoid having to keep querying the database in hopes of obtaining a valid cursor. So here with all the pieces in place (AwaitData, dummy document, and sleep before query):

using dummy document

This produced a standard deviation of 0.000662s, and never spun the CPU. The final c++ client code to produce this result (just the outer while loop):

while( true )
{
     auto_ptr c = conn.query("queues.messages", query, 0, 0, 0,
             QueryOption_CursorTailable | QueryOption_AwaitData);
     while( true ) {
         if( !c->more() ) {
             if( c->isDead() )
             {
                 // this sleep is important for collections that start out with no data
                 sleepsecs(1);
                 break;
             }
             continue;
         }

         const BSONObj& o = c->next();
         lastId = o["_id"];
         const double time = o["time"].Double();

         // handle the dummy document {time: 0.0}
         // yes, not a great comparison for doubles, just an example
         if (time == 0)
             continue;

         gettimeofday(&tv, NULL);
         const double curr = tv.tv_sec + tv.tv_usec / 1000000.0;
         cout << i++ << ", " << curr - time << endl;
     }

     // prepare to requery from where we left off
     query = QUERY( "_id" << GT << lastId ).sort("$natural");
 }

You can see how I bypass the dummy document used to prime the cursor as an example. I ran a similar consumer/producer test with no time delay on the producer (producing ~24,000 msg/s) and also obtained similar results with a standard deviation of 0.000753s.

Conclusions

Given the above latency measurements, I found that MongoDB tail cursors and capped collections performed much better than I expected. I was hoping for latencies in the 10 to 100 millisecond range but instead often got sub millisecond latencies for standard deviation with peaks of 5 milliseconds for the last run. Depending on your definition of “low latency” using MongoDB for basic message passing is quite viable and gives the added benefit of logging all of the passed messages for things like replay or catch up (if you have sequence numbers).

In performing this benchmark I encountered some pain points. The biggest one that may lead users astray is the incomplete example on the mongodb.org page (as of this writing). It does not mention the very necessary AwaitData flag, and also incurs a high latency due to the use of sleep.

Another problem (and something I consider a bug) is the behavior when there are no items in the collection. Depending on your use case and consumer/producer startup times, this may be a hidden issue that is hogging CPU time until a valid cursor is created. From my point of view, a valid cursor should be created immediately (even with no documents) that can then behave properly under the AwaitData condition.

And finally,  it would be nice to expose an API that allows the c++ client to epoll (or some other notification scheme) when new documents are available. Since with the AwaitData call the “more” cursor method blocks for a bit, this becomes very unsafe in event driven systems which should not be blocking (or any system that should not block). Since there is a socket underlying the connection to the DB, this seems like a reasonable addition that would solve such a problem.

Additional Notes

All testing was done on a Core i7 920 running Ubuntu 10.10 and using SSD as the storage medium. I used the latest of the MongoDB master github branch as of June 9th.

Initially, I tested a simple python consumer, but it too suffered from the sleep problems and did not expose query options in the same way the c++ driver did. I stuck with the c++ driver as an example given the clarity in demonstrating the various “gotchas” in doing the benchmark.

I toyed around with the node-mongodb-native driver and think that the “streaming” aspects could make the callback API work a bit better but would need to be flushed out more with how the “GET_MORE” operation to the server is handled. I found that I could lock up the server if I was not careful with how many GET_MORE ops were being sent for the cursor.

So why the change of heart on tipping (if you can really call it a change of heart)? When I wrote my first post on tipping, I was completely focused on the idea of tipping as something “extra” beyond the basic salary of the individual. I viewed tipping as a failure in that it was diluting good service (I will come back to this later) and basically making tipping become the “norm” or “expected” thing to get any service at all. I felt that I should be able to pay for a service and get that service performed (to the ability of the person) and pay for the service given a fixed and known cost. What I failed to realize was that tipping isn’t about the server getting money beyond what they should be making but that it is actually a way for the consumer to rate the server in a dynamic setting. Not giving a tip is the equivalent of a bad performance review (sort of like a negative tip if you will); and by tipping the “social norm” you are in effect paying the market price for the good. So a tip above the social norm is one that commends good service. By tying a portion of the salary to tips, the server and consumer can create a tighter feedback loop and ideally improve service faster for everyone. While there is the possibility of complaining to the cab company about poor service (or police if something illegal is happening) these type of things take time and are not always as clear to the driver who may benefit from more direct feedback.

However there was one thing that didn’t quite sit well with me and that was how the “social norm” number comes to be determined or even what it means. I could believe the argument about lower tip equating to bad performance, but how much tip equals “standard” performance (if  there really is such a thing). I felt strongly that even without a tip the drivers should be doing their jobs to their ability and that a tip should not warrant normal service. Unfortunately, it was pointed out to me that if all drivers knew they had fixed income and no tips, then there would be less incentive to perform better or give good rides. I will not tackle the social implications of this or my thoughts on it since I already conceded that I liked the idea of a tip being like a small performance evaluation; I just needed to know what that should be.

It is common knowledge that waiters/waitresses do not make minimum wage because tips are expected to bring their salary to the right level (again… how you are supposed to know how much to tip to make that happen is unclear); was the same true for taxi drivers? I figured that given the higher skill, longer hours, and more exhausting working conditions that taxi drivers would be making at least minimum living wage without tips. I found a website that listed the minimum living wage in NYC at $10 with benefits and $11.50 without [1]. Another amazing source (Taxicab Fact book), which I highly recommend browsing, listed that the average hours worked on a shift was 10 and that a full time driver (only about 40% of all licensed drivers) reported working 5 of the previous days [2]. Lets say that a full time driver works 5 days a week for a 10 hours a day. Given the requirement that they should at least make living wage that would mean $11.50 * 10 * 5 * 52 = $29,900. For living wage a full time driver should be making right under 30k a year. Clearly driving a car (especially well in NYC) requires some skill (we are not going to debate how much) so it would make sense that such a job would pay more than minimum wage, does it? Indeed it does! The average NYC cab driver salary is $49,532 [3] just under 50k. So clearly on average cab drivers are making more than the living wage. But, the interesting thing I found about this number and other wage estimates for cab drivers is that they take into account a 15% tip for fares. That’s right, built into the expected salary is a 15% tip. This means that without tips, drivers make slightly over 43k a year. This 43k comes to about $16.50 per hour given the hours I calculated before. Compare that with a rough estimate (google) for MTA bus drivers making about $26~$29 per hour. To me it seems reasonable that cab drivers make less, but not much less. Indeed at 50k a year (after putting the 15% tip back) they make closer to $20 per hour.

Now there isn’t a hard conclusion I am drawing from this; or saying that drivers make too much or too little. I was just interested in getting some rough numbers for comparison and finding how the “social norm” tip rate figured into the calculations. This just means that to calculate the real taxi rate you just need to add 15% to the posted rate. Ideally the “social norm” rate doesn’t change (as it seems to for restaurant servers) and thus allows the taxi companies to better price around such a rate. Why 15% was chosen versus 10% or some other arbitrary number I couldn’t say (at least not with way way more information). Given that NYC trips are pretty short and come in at under $10 [2] it would not be a hard leap to think that a smaller tip would just not carry the same weight.

Overall, I found the various articles interesting reading. The taxicab fact book [2] and a more dated (1995) overview [4] from the same company were by far the most informative. The cover such things as accident rates and how fares have changed over time. In the end, I feel comfortable settling on paying the “social norm” tip given that it figures in the income rate and is documented at a known 15%. Given that it stays constant at this percent, it is perfectly reasonable to treat it like a performance bonus rather than a specific tip.

[1] http://www.livingwagenyc.org/pagedetail.php?id=3

[2] http://www.schallerconsult.com/taxi/taxifb.pdf

[3] http://411newyork.org/guide/2008/03/30/new-york-taxi-cab-driver-salary/

[4] http://www.schallerconsult.com/taxi/taxi1.htm

Before you start to get all defensive and claim that the cab drivers depend on the tips for their wages and that they provide a service and you tip for a service..etc etc let me clarify my position and also state that me not tipping doesn’t mean you have to not tip; it’s your choice!

Sure, if the cab driver provided you with an abnormal taxi experience, or somehow otherwise stepped outside the “driver” role, then feel free to tip away. On the other hand, if they just picked you up, and drove you to your destination, then I would not tip and here is why. The product you paid for was the ride and the driver is a part of the product. Without the driver there is no product. I view this differently from a true service industry (like food) where you consume a product separate from the service provided by the staff. When I tip in a restaurant, I don’t just tip my waiter directly, I take into account the support staff that make up the entire experience (even though I know my food costs have taken that into account to some extent).

Another thing about cabs that differs from a normal service experience is how the costs can be figured in. In a NYC cab, the price is listed as per some fraction of a mile OR time elapsed. This means that the cab driver’s “time” can be figured in just as much as the operation costs of the taxi itself. The longer you sit in a cab, the longer the cabbie “serves” and the higher your fare goes. it grows with the time of service (or distance of service). The fact that there is an additional tip often given is just encouragement to not price the fare correctly. But I am sure the cab companies know the costs and have figured them into account.

I take the same stance on other services which cannot be separated in a meaningful way from the service giver, like a haircut for example. The haircut (product) and person doing the haircut cannot be separated, otherwise you won’t be getting a haircut anymore. Various haircuts have their own levels of complexity and can be priced accordingly (and often are). The stylist (or barber) will know the time taken for the haircut and effort involved in the requested style. But you have to be careful about future service investment.

Future service investment is the idea that by tipping today and being a repeat customer, you will get good service next time. While some may sneer at this concept, it is something valid to take into account when determining your tip amount. A taxi ride has a lack of future service investment with respect to tipping. By not tipping you do not run the risk of ruining the service for your next taxi ride simple because the chance that you will encounter the same taxi driver for your next ride are… well, small (discounting custom limo services or the like with a limited driver base). This means that (unlike a resaurant you frequent) tipping a taxi does not “invest” into your future with the taxi company. You may begin to think that you ruin the experience for the next person and so on, but you don’t. The driver will most likely shrug you not tipping off as a nothing, or better yet make an effort to be a better cabbie for the next person who will most likely provide a tip. But lets say that many people stop tipping cab drivers, then either the cost will rise to reflect the “true” cost (which I am confident is already in the price) or the cost won’t rise and cab drivers will no longer expect tips and drive as usual. Think of yourself as a foreigner who (probly well and good knows about US tipping customs) but still doesn’t tip because… well who cares, they won’t be back).

While my overall views on all tipping are negative, mostly because I don’t think it should be a socially common thing to do, I acknowledge that some jobs and wages are constructed with the idea of tipping in mind. Mostly, I don’t like to promote the idea that the price listed is not the real price. If there are costs involved, then those costs should be figured into the price of the good being consumed. But if the time and effort spect cannot be well predicted in advance, then you do have a genine opportunity for some tipping (not the expreme society has come to accept as the norm). Don’t always tip just because there is an are on the receipt for “tip” or because the option is there. Tipping should be something that is above and beyond, not boring and ordinary, otherwise, the whole notion is diluted.

www.socialsecurity.gov/onlineservices/

Check out the left side. The “Field Office Locator” is only available during certain hours. Note, this isn’t even a chat feature or anything like that… its a simple lookup service. On the right side, there are even more hours posted for what appears to be the site in general. I’m not sure why this is. Maybe they run out of RAM on a known schedule. Anyhow… no longer will I “assume” the internet is open.

When messing around with the some graphics one night I decided to dedicate some time to seeing how easy it would be to make a new kdm theme that would work a bit better with the ksplash and wallpaper that will be used in Kubuntu Lucid. After about an hour or two of messing around and pulling artwork from different places then tweaking it a bit I came up with:

Kubuntu Lucid KDM Theme

This will be the new default theme for Kubuntu in Lucid (10.04) when it releases. This will also be the default wallpaper. the main goal with the theme was to make the transition from kdm -> ksplash (the splash screen showing kde loading progress) a bit smoother. This is accomplished by using the same center box graphic and wallpaper scalings that ksplash uses. Overall the effect is nice. I created a short [exciting] video to show off how it looks. Apologies for the sound… I had some recording difficulties…

The problem

Imagine you are a new Ubuntu user. When you first encounter a problem, it may be very difficult to get your bearings and figure out where you need to go to get the best support. Most users will try to go to Google and search there, likely with limited success as they will not know the best query to use. Other users will navigate to Ubuntu.com and hopefully see the small “Support” link at the top. The user can now select from three choices: free documentation, paid support, and free community support. Let’s say this is a single user just trying things out and not really interested in paid support. This leaves them with two options.

The first option (documentation) takes them to pages where they have to click through lists and lists of various “categories” trying to find the topic that pertains to their problem. Granted, this documentation is up to date and of good quality, but again might not be the most welcoming environment to someone who is frustrated with a single problem and doesn’t know where to go.

This now brings me to the community support options. From those the user can pick any of: live IRC chat, web forums, mailing lists, and the answers system. Notice that for a user looking to solve a particular problem they now have to hunt through 4 very different systems for a solution. Mailing lists would be difficult to search and, for a new user posting to a list, would not necessarily be the most pleasant experience. There is also a signup process with posting to a list and responses are not immediate. The list topic can bounce around for a while until some conclusions are reached. The web forums will probably look familiar to many users. But again, you have to sign up. After that, you post your problem and check back (or get emailed) if someone responded. Then you respond back and this pseudo email exchange goes on until … well… who knows, hopefully a solution is reached (potentially days later). The user might then try the answers system. This system is geared towards just that, answering questions for users. A user posts a question, and others can comment on it and hopefully guide the user in the right direction. The different between answers and the forums is that answers live on Launchpad and can be linked to bug reports easily. But essentially the two are very similar. You post, and you wait.

The best “live” solution the user can pursue is to use something called IRC (Internet Relay Chat). For a new user, who might be very familiar with the idea of instant messaging, will most likely have never heard of IRC much less used it. Although the instructions are quite clear on how to connect (join a server, and go into #ubuntu, or #kubuntu…etc). Once the user is in that channel they are in what is basically a free for all. Being brave, they will post their question to the channel. It will be mixed with all of the channel traffic already there and hopefully seen by someone who can help with the problem. That person might give a few pointers and others might chime in at random times. Oh, and don’t forget that other channel conversations are still going on at the same time. A new user would be completely lost in this wave of messages (and probly not know common idioms like prefixing a message with the username of the person it is for). Hopefully the user will not become more frustrated that their question isn’t getting as much attention or that they can’t figure out what is going on in the channel. The user will get help, but who knows how pleasant an experience it will be for them. And once they do get help, the IRC logs are there but searching through them for that one user’s problem would be a nightmare for anyone wishing to follow-up. But! The user did get “live” assistance when they needed it most.

Hopefully by now I have shown you that for a new user, finding out where to get support and then actually getting that support in a reasonable manner is no small task. Yes, a person that is familiar with each system will obviously know how to find what they need or use each properly, but we have to approach this from the standpoint of a user who does NOT know.

My Solution

I now come to my actual idea. It originally started out in my mind as a simple one-on-one chat client but has since grown to hopefully be a one-stop-shop for a user in need of assistance and guidance. I will try to explain the evolution and reasons behind the idea.

A user has a problem and what’s to find the most efficient way to solve it. One of the fastest ways we can do this is to connect that user to another Human Being(tm) so they can “talk” about it. I know, this whole idea of actually having someone talk to someone else in real-time sounds crazy, but work with me. The feel of such a client would be minimalistic; think a simple one-on-one chat client. With one-on-one the new user would not feel as intimidated as in a massive IRC channel. Plus, they don’t even *have* to know it is an IRC channel. All they care about is that some is helping *them* with *their* problem; even if that someone is doing something as simple as guiding them in the right directions.

The workflow would be something as follows:

1. User launches the live assistance application (it is already installed on their computer).
2. They type their question/problem into a dialog.
3. A chat window comes up notifying the user that their question has been posted and that hopefully someone will help them shortly.

That’s it! No hunting through documentation, no new systems to learn, just a familiar IM chat interface.

“Well hold on there!” you may be wondering. “How can we ensure that the user gets help?” You ask. Well, we can’t I answer! But is this any different than the other methods I described previously? In all cases the user put their question out there and some kind souls decided to respond. Similarly, in this case, the client put the question out onto one of the many existing IRC support channels where people are listening 24/7. Another user on that channel sees the question and opts to start helping the user with their problem. The beauty here is that no party (supporter or supported) had to go anywhere new. The person on IRC stays on IRC and the person on the desktop didn’t need to stray anywhere new. The user with the problem didn’t have to hunt for anything.

This was the first part of my plans for such an application. Already I have a basic implementation in place for the chat client and supporting bot. But there are more things that can be done to extend this basic idea and really make it a big selling point for new users.

The first thing would be to give the user something to do while they wait for assistance. This could simply be a bot that tells the user about the various existing support options and provides links to each. This information would be presented to the user in the chat client so they would not need to go anywhere to hunt it down. In the future, the user’s problem may be queried against the forums or answers system to present them with possible things to look at while they wait. For now, the focus remains on having the user talk to someone that can help them.

Since all of the individual help sessions are logged. A simple web frontend could be created to be able to search through and even see which problems users are facing. These sessions could be tied to bug reports (much like answers are now). Both developers and usability people could benefit from the data as it would outline the actual process a user took to fix something or show holes in understanding.

I feel that having such a centralized, human centric, and user focused system is important for a large scale distribution which prides itself on being “Linux for human beings”.

Details and cool screenshots to come soon