Organizing a Workshop
1 TL;DR
2 Basic Rules
3 Planning by Survey
3.1 Generic Survey
3.2 Specific Surveys
4 Talk Advice
5 The Icebreaker
6 Day Schedules (Dagstuhl)
7 Publications
8 Nurturing the Seedcorn
Monday, September 21st, 2015 4:10:01pm

Organizing a Workshop

Shriram Krishnamurthi

    1 TL;DR

    2 Basic Rules

    3 Planning by Survey

      3.1 Generic Survey

      3.2 Specific Surveys

    4 Talk Advice

    5 The Icebreaker

    6 Day Schedules (Dagstuhl)

    7 Publications

    8 Nurturing the Seedcorn

Versions: 2015-09-21. 2015-02-20 (with extensive feedback from Michael Hicks). 2015-02-07. 2014-11-08. I’ve been fortunate to jointly run a few nice workshops (and a few weak ones, too). I’ve also attended about thirty ad hoc events whose organizers were charged with preparing a one-off schedule. I’m writing down some of what I’ve learned from all this. These instructions are especially designed for venues like Dagstuhl: small, focused, and enclosed, with high quality-control on who participates.

1 TL;DR

There is one overarching principle that covers everything else below: Take Responsibility. Perform the following calculation. Say 40 people are attending your event, at an average cost of $1000 per person (almost certainly a very weak lower bound). All told, a 5-day Dagstuhl is roughly a 4-day event. So the workshop is costing (probably mainly taxpayers somewhere) $10,000 per day. Does each day deliver $10,000 worth of value? When you can say “Yes”, you’re done planning; until then, you’re not.I perform a similar calculation every semester for my courses. It’s sobering.

It’s exciting to submit a workshop request. It’s even more exciting to get an acceptance. However, unlike a conference paper—where your work is mostly done at that point—the point of acceptance is when your work actually begins. Don’t bask in the acceptance for more than a day (if even that long); get cracking.

Good luck. I hope your event is a huge success. If you have comments about the following content, please contact me.

2 Basic Rules

Here are some important rules to keep in mind:
  • Remind people they are there to participate, not necessarily to speak. There’s an important difference. If you make the latter the focus, some people will simply show up, give their talk, and not get involved in anything else. Too many of these people and you end up with a room full of people reading email most of the time. If you make clear that they may not get to give a talk—but can still participate, usefully—it not only sets the right tone, it weeds out attendees who really had no interest in the event being a two-way street. Hopefully they won’t come at all.

  • At the same time, recognize that people want a chance to be heard. They may even need to formally be able to show this to get their trip reimbursed, etc. There are ways (The Icebreaker) to get most people to speak a little early on. Once they’ve had a bit of a say, they will be more attentive to what others are saying. Otherwise they will obsessively fine-tune their own slides, shutting out everyone else.

  • Control the WiFi. Tell everyone that you have the ability to turn it off, and will do so if you find that too many people are spending too much of their time reading email. However, don’t turn it off by default. Not only are they adults, they may have perfectly legitimate reasons (e.g., downloading a paper corresponding to a talk they just heard); besides, if sufficiently many of them are bored, you need to get your event back in shape first.

  • Don’t operate out of a mistaken sense of “fairness”. Not everyone needs to get the exact same amount of time to speak (with zero being the limiting case of this). Fit their topics to the audience. Within limits—that is, without abusing power or going overboard—remember that you are more like a creative director or composer, not a mere bureaucrat dividing out minutes with emotionless arithmetic. It is perfectly fine to have everything from five minute to one hour talks, in various distributions. In fact, mixing things up a bit can keep an event more lively.

  • Take into account travel and time differences. We schedule events as if our bodies can magically take the abuse of a ten hour flight, jet lag, dehydration, and so on, and function exactly as they would at home—and then ourselves suffer from the consequences. If you have significant populations crossing many time zones, think about when they are likely to “crash” and how the schedule can accommodate that. For instance, don’t put an hour-long talk at that time; have lots of short talks, or a poster session (which lets people walk around), or even a group activity.

  • Give people time to exercise or seek other outlets. At Dagstuhl, in particular, there is a tendency to run events all the way right up to meal times. For people who want to do something physical, doing so after a meal may not be easy. At Dagstuhl, stop the day at 4:45pm so people have time to return to their rooms, change, exercise, go back, shower, and still make it to dinner on time.

  • Leave some time for people to be on their own. It’s a sad fact of our lives that we’re flooded with email and other interruptions that seem to demand constant nurturing. If you leave some room in the schedule, you can convince people to put off their emailing to those times. If you don’t, they will eventually feel this intense pressure to ignore the event and switch to their mailbox.

3 Planning by Survey

My style is to run a few quick surveys to plan the event. Keep the surveys brief to maximize participation. If you run them too early, nobody pays attention; if you run them too late, you give yourself no room to make adjustments or plan creative events (e.g., inviting certain people to prepare talks that the audience wants to hear).However, don’t put too much faith in survey results; use your judgment too. I have sometimes scheduled sessions that I thought would be dreadful but that got the most votes, and by half-way in the whole room wished the session would end. Starting about three months out seems just right, because by then your attendee list will also have settled.

To run a survey, you may find it convenient to use various free on-line tools rather than doing it all by email. I have found SurveyMonkey to be perfectly good for most purposes.

3.1 Generic Survey

This survey is the most important component of designing your workshop. Keep the message focused so the recipients don’t lose sight of its purpose: you want as many people as possible to respond. Remind them of the purpose of the workshop, and then ask:

  1. Your name:

  2. What would you like to hear about for 5-10 minutes? (use generic topics)

  3. What would you like to hear about for 30 minutes? (use generic topics)

  4. What topics do you think you can speak about for 5-10 minutes?

  5. What topics do you think you can speak about for 30 minutes?

  6. Do you have suggestions for things we should do at the workshop (besides keeping time free for people to mingle and to organize spontaneously)?

  7. Do you have suggestions for things we should not do at the workshop (besides not cramming it full of talks)?

  8. Any other thoughts/comments/suggestions?

You will need to cluster their responses to find both affinities and outliers. (Neither is more important than the other; in particular, the outliers may be very specific people and interests that are not more widely known, and maybe you should give them a little more time than you do those that are already central to the area.) Naturally, people won’t use the same generic terms, so you’ll have to use your judgment and expertise as you classify and cluster. Knowing who they are not only helps you figure out whom to call on for what purpose, it also lets you email them to figure out their intent behind some of these answers.

Michael Hicks suggested another way to think about these questions. You could instead ask questions like:
  1. What are things you feel prepared/comfortable talking about?

  2. What are topics on which you would like to get feedback?

This is an orthogonal dimension of questions to the one above. The important division the above two questions get at is: the former is about the audience learning from the speaker, while the latter is about the speaker learning from the audience.

You will spend a lot of time processing the output. The quality of your event will depend heavily on how much attention you pay to these responses.

3.2 Specific Surveys

Keep in mind that there are hidden interests not revealed by the above survey. For instance, there may be many people interested in the topic of exploding pumpkins, but few thought to mention it. But when you put it on a menu of options, it may percolate to very near the top. Therefore, you should view the clusters you formed above a prompts: now you present them all to the group and gauge what their interest level is in each one.

There are various other specific, topical questions you might wish to ask. For instance:
  • You might want to organize some debates; for each debate topic, find out whether there are people with strong views on it, people willing to take a lead in a discussion, and whether people even want to hear a debate on it.

  • You may have more topics than time, in which case a survey can help you identify which events you can most afford to have run simultaneously.

  • You might invite people to submit specific puzzles or challenge problems. These are often best done as two brief sessions with some time in-between: a brief presentation early in the week to present the problem, and a longer session later in the workshop to discuss solutions, approaches, etc.

  • You could even envision an entirely different workshop structure. David Eppstein says:

    “The workshops that I’ve been most satisfied with have […] been focused around doing research: collecting problems near the start of the workshop (maybe with a small number of overview talks) and then spending the rest of the time in dynamically changing working groups trying to solve these problems (with periodic progress reports and the usual social activities).”

    You might be able to kickstart such a process with a preliminary survey (or at least use a survey to determine whether this style of workshop is feasible with the attendees you have).

And so on. So long as you keep your subsequent surveys easy to take (few questions, all can be answered with a check or click, comment boxes are present but optional), people will respond.

4 Talk Advice

Inevitably, a fair bit of time at your workshop will probably be spent hearing talks. This isn’t inherently a bad thing. At a one-off event like a Dagstuhl, this is a rare opportunity for a group that might never congregate again to hear ideas from outside their ambit. So don’t excessively undervalue talks in favor of other snazzy activities.

That said, it’s important to get good talks. Therefore, the most important thing you can do is to encourage teaching. Many of your participants are likely academics. They teach, and some probably teach very well. When they teach, they likely often do it by writing on a board.I believe slides are the enemy of good teaching, but that’s a topic for another day. Put them in a “conference stage” setting and they become all stiff, formal, and one-way; put them in a “front of the classroom” setting and they become loose, casual, and bidirectional. (You, Dear Reader, may even recognize yourself in this remark.) Encourage them to skip slides entirely and use the board. Chances are it may have simply never occurred to them to do this, and even if they don’t take you up on your suggestion literally, it will get them to think harder about how to present their material.

The next most important characteristic of quality is to avoid canned talks. The speaker’s most recent (or old) conference talk left completely unchanged is usually a pretty bad candidate. You have to get them to spend a little time customizing their talk for the venue.

In particular, tell speakers to edit out the obvious material. (You’d be surprised how many need to be told this.) For instance, don’t come to a Web security event in 2014 and begin with a whole slide on the importance of JavaScript. Don’t come to a programming languages event in 2014 and begin with a whole slide on the rise of Python. Don’t quote Gartner reports. Assume that your audience has basic knowledge and smarts.

As a stick,There is some disagreement about the precise meaning of carrot and stick: are they two different things (an incentive and a punishment) or two parts of a single system (a carrot dangling at the end of a stick before a donkey: a seeming incentive that can never be attained)? I think of it as the latter, but the former use seems to have won out in practice. threaten that the entire room will hoot and holler if they put up this kind of “throat-clearing” material. As a carrot, offer to work with speakers to figure out what needs to be said and what doesn’t, and really do spend some time with them on this. Nobody should even notice the missing content; that’s when you’ve done your job.

Be on time. It’s so obvious it barely needs stating, but it’s important. Few things agitate people as much as a sense of unfairness. An early speaker in a session who takes too long is being unfair by putting the squeeze on the later speaker. It ruins the latter speaker’s talk, which may have been carefully tuned.

Marc Herbstritt pointed me to a Dagstuhl timer. This could be useful, but I warn against being too rigid. Sense the temperature in the room and, if a reallly good discussion with broad participation is going on, it’s okay to let things run over a bit. This is, after all, a workshop; the whole point is to explore things. Therefore, build slack into the schedule (if you don’t need it, well, nobody ever objects to finishing a few minutes early). Remember that you’re a conductor, not a metronome.

Tell all your speakers to have their talks ready before the event begins: that they may need to be able to go on very short notice. (If they’re still working on their talks there, then they can’t be paying very much attention to other speakers, can they?) This way, you can accommodate scheduling changes better by moving people around dynamically: e.g., if your last speaker of the session had 15 minutes but earlier talks ran over (with your permission) by 8-9, grab a 5-minute speaker from later in the event and move them forward; after the session finishes, rearrange things.

There are two corollaries to being on time. One is to know how you’re going to keep track of time. Make sure all session chairs have some means of timing. The other is to select session chairs whom you trust to appreciate and enforce this matter.

5 The Icebreaker

One of the problems that vexes (or ought to vex) organizers is how members of the group introduce themselves to one another. Since not everyone will necessarily give a talk, and even if they do it might not be until late in the week, that’s not a very good option. What else to do?

A staple is to start with a half hour when everyone stands up and introduces themselves, a ritual for which there are no known fans. I can’t make out half the names of the people. Second, some people go on for too long, and you can sense the tension in the room as this happens. Third, if you’re actually paying attention, about ten people in it’s pretty hard to keep track of the endless stream of content. Finally, the introductions are either perfunctory or they sap energy. (If you are going to do this anyway, ask people to pronounce their names and affiliations slowly and to keep their self-descriptions focused. This is a time when the anxious will try to enumerate every single thing they or have ever done.)

Fortunately, there are at least two good alternatives.

6 Day Schedules (Dagstuhl)

Some of this material is just common sense, so you should be able to adapt it to other venues than Dagstuhl.

If you’re running a five-day Dagstuhl event, you have sixteen 1h30m sessions at your disposal: four each on Mon, Tue, Thu; two on Wed; and two on Fri. That’s the theory; now let’s look at the practice.

First, you don’t have 1h30m; you actually have about 1h15m per session. If you get the arithmetic wrong, that’s a lot of minutes to make up.

You should assume that people will start drifting away on Friday: at Dagstuhl this could be Europeans who can drive home, Asians and Americans who want to take a midday flight, etc. In general, it makes sense to wind things down by around 11am on Friday, let people socialize for an hour, get to lunch in leisure, and leave.

The evenings are a tricky time. Sometimes it makes sense to organize an evening event (such as a software demo), but make sure it’s optional and labeled explicitly as such. Younger attendees, especially, may not realize what is okay to skip, and as a result may both deprive themselves of great social interactions and end up exhausting themselves before the week is over.

Try to have a mix of events. Some will be sessions of talks, some might be debates, some may be brainstorming activities. Try to avoid the generic “breakout session” without a clear plan of what you’re breaking out about. This doesn’t have to be a goal—the main purpose should be to talk, not to produce output—but you should know what the subject is going in (even though it may not be the subject coming out).

Don’t over-schedule the week. I suggest having a schedule that runs through lunch on Thursday, but with the understanding that everything from Wednesday onward is flexible. By Thursday if you haven’t found things people want to talk about more, something’s wrong with the event. If you have, then you have plenty of time to figure out how to structure Thursday afternoon and Friday morning. Use the earlier part of the week to accumulate topics that people might want to talk about later. Don’t forget to schedule the The Icebreaker.

At the same time, don’t under-schedule the week either. The very worst kind of event is one where organizers say they will figure out the schedule “at the workshop”. To me this is just a euphemism for “Despite being called organizers, we couldn’t actually be bothered to organize.” These events are often terrible unless the organizer has some brilliant plan for how this will all come together (which hasn’t yet happened on my watch).

If you’ve done your job, you have various other organizational tools available. For one thing, you could introduce people by clusters. Another option is to have just those who aren’t speaking in the first two days give a very brief summary about themselves, but let their talks speak for the others. I suggest doing this not at the very beginning of the first day. Instead, get straight to something with content and interaction.

7 Publications

You will probably be expected to write a report. A useful technique is to ask the graduate students and other young scholars to serve as scribes. (Of course, don’t over-burden them so they can’t participate themselves.) It has sometimes been effective to assign a pair of students—a master scribe, and a reader—to each activity (session, breakout, etc.), with the scribe responsible for producing the drafts and the reader reading drafts to help improve them before they percolate up to the organizers.

Beyond such a report, unless you have to (or that is your express purposeAn obvious exception would be a Dagstuhl Perspectives type of event.), don’t make the event be about producing a document, such as an LNCS issue. If you are going to, make that amply clear up front—at invitation time—so those who are not interested in such an activity know not to attend. And then, organize your entire week around the production of these documents.

It’s tempting to ignore this last bit of advice and try to have it all. Remember that it’s very easy for people in the heat of the moment to salivate at the idea of yet another publication and say yes to just about anything you ask for; the odds of actually getting something out of them once they return to their routines may be close to zero.

It’s okay to not have a written record. It’s okay. It really is. Science is a social activity, and we should be honest about the value of that socialization instead of having to always cover it up with reports that might be read fewer than one time.

8 Nurturing the Seedcorn

Pay attention to younger members. Know who they are; know who’s vouching for them. Make that person do some work to prepare and nurture this person. (If they’re someone else’s advisee and the advisor won’t be present, that mentor may very well be you. As the old line in a slightly different context goes, if you’re looking around the room to figure out who it is, you’re it.)