conjugateprior

How to run a course (if you're me)

Last summer, I and my trusty henchpeople from the Department of Politics ran an intensive six week summer course for incoming freshmen on data science ('POL245', for locals).

This post sketches out how I think course infrastructure should work, and provides some practical details of how we arranged things. Most of our structures worked pretty well. Some didn't, so I also try to say what we'll do differently next time. Some of this is, inevitably, specific to POL245, but maybe there's something you can take away if you ever teach such a course yourself.

Managing students

When undergraduate students are faced with uncertainty about some aspect of a course they take a random walk through their information sources until they find some kind of answer, then run with the result. This means that any inconsistency in materials will eventually be discovered and lead, at best, to some proportion turning up late or in the wrong place, and at worst to arguments about grading or attendance.

The solution is consistency and incomplete contracting. The best way to enforce consistency is to remove information source duplication and automate as much as possible. As many random walks as possible should as soon as possible lead to the right answer.

Incomplete contracting is a larger topic than I can deal properly here, but here I mean roughly that ambiguity (due to lack of specification) is better than inconsistency because it allows the possibility to set expectations mid-course without compromising existing expectations about how the course works.

Managing course personnel

In most courses, most of the time, both instructors and assistants are doing other tasks at the same time, e.g. writing theses, doing research, and planning or teaching other courses.

Good course infrastructure should therefore do two things:

1. minimize the cognitive load of being involved in the course
2. maintain morale

Cognitive load minimization is a matter of encouraging course personnel to have have regular expectations about the course; just like for the students really. Teaching a course should not feel like fighting fires.

Morale is maintained by engineering suitable social infrastructure. More on that below.

I’m no management theorist (and by all accounts a mediocre manager) so if much of this sounds naive, then that’s probably because it is. Give me a better way to think about it. In the meantime this is more case study than lesson or recommendation.

Course organization

Each week of POL245 consisted of: 2 lectures; 2 tutorials, here called 'precepts'; 3 help sessions, here called 'quantlabs’; and 3 sets of office hours. There were also 4 take-home homeworks, called 'problem sets', one final project presentation, and no final exam. That’s a lot of stuff for everyone to keep straight.

What we did

A syllabus containing course information was distributed on the first day of class. (Latex source, distributed as PDF)

We also kept a course calendar as the day-to-day source of information for where students and course personnel need to be and when they need to be there (hosted on Google). The calendar contained lecture topics, room assignments, problem set handing in times.

What worked

It was nice to be able to say with assurance: "it's all in the calendar". Oddly, our students were more likely to consult a calendar than the syllabus.

What didn't

Consistency: Keeping multiple information sources consistent was, as always, a persistent challenge. Here there were only two, but that’s one more than we’d ideally have.

When things changed - basement classrooms flooded, rooms got rearranged, precept topics got changed - the calendar was a lot easier to change than the syllabus. Consequently the syllabus and calendar inevitably desynchronized as the course progressed.

Transparency: Despite invitations, it was unclear whether our administrators and thus other parts of the Summer Program could see what we were up to.

What we’ll change

Consistency: In the future we'll keep the unchanging course metadata - mark scheme, textbook listings - in a syllabus document, and try never to change it. We’ll put all the day-to-day course details in the calendar.

Transparency: Our institution operates on Exchange [booo...] so we will, with some regret, probably move our calendaring there next time around. As with all coordination problems, sometimes you just having watch the boxing...

Course technology

We used Blackboard to post lecture slides and course announcements, mostly because the students will have to wrestle with it for the rest of their courses and we wanted to warm them up. Blackboard seems to have one advantage - timed release - and disadvantages too many to usefully enumerate.

What we did

Lecture slides, precept materials, and problem sets elements were revealed to students on a timer and completed problem sets had to be uploaded by a preset deadline. Timers were set before the course began.

What worked

Setting problem sets and materials before the course began to appear at the appropriate moment worked very well. In particular, Blackboard makes it possible to set a file download to appear without requiring the actual file that will be downloaded to be present yet. Being able to set this up this before there were even any materials to post was nice.

What didn't

Blackboard is not a particularly pleasant way to arrange these things but it is what the institution provides.

Technical friction: Our instance didn’t allow uploading of HTML files directly. This was unfortunate because problem sets were handed in as rmarkdown compiled to an HTML page. More on that later. Consequently, students had to zip up their homeworks and unload the zip file. This was a waste of everyone’s time, but did reveal that most students and one tutor didn’t know how to get files into or out of a zip archive.

An interlude on students

It's common to say that incoming students are 'digital natives'. This is true, but not in the way you might expect. Most of us quantitative social scientists have the same relationship to our students as field linguists have to native speakers: The natives operate more or less fluently in technologies that we had to learn but, being socialized into it, have no idea about how or why the technology works. Like native speakers flummoxed when presented with a strange construction in their mother tongue, our students fail to generalize because they do not see the alternatives.

Take a concrete example: our students usually lack an explicit understanding of the 'file-folder' metaphor of computer operating systems, so we have to teach it. This is not surprising. Our students may never have physically filed anything, and their phones have a very shallow implementations. Moreover, contemporary operating systems mess with the simple tree structured file system concept in a way that makes abstraction harder: Windows has multiple file roots and shows the content of as yet unzipped archives the same way as actually unpacked files (yes, I know why, but it's an abstraction only a computer scientist could love). OS X does not include a user’s home directory in its file navigation tool by default, but does including an virtual ’All Files’ folder from which things cannot be zipped, and all location information has been removed.

A substantial part of this course is therefore introducing students to their operating systems. Almost all our students left everything in their Downloads folder (as soon as they managed to find out where it was) and just worked there. That was just the least bad option.

Despite these annoyances, Mac users had a huge advantage (as they do in any course involving computers.)

What we’ll change

We'll stay with Blackboard. It took me, a Blackboard neophyte, about 2-3 hours to set all the structure up before the course, and about 0.5 hours per week to upload materials before the links were revealed. Not too bad, all considered.

We’ll add an afternoon session before the course begins in which I and perhaps the tutors will introduce students to their operating systems.

Course personnel

POL 245 has a rather involved personnel structure: There is a 'course head' whose role is presidential, an 'instructor' (me) who gives the lectures and is in charge of day-to-day course management, three 'preceptors' who lead the course tutorials, and six 'quantlab tutors' who run the help sessions. In the background, two administrators oversaw this and other courses as part of the summer program. Lots of people doing lots of stuff. How to manage it?

Communications: What we did

Communications In previous years POL 245 generated mountains of email traffic. This year I moved all course communications onto Slack and set up the following channels:

• #calendar an automated channel with update messages triggered by changes to the course calendar
• #dataissues discussion of student problems acquiring data for their final projects
• #general course announcements by the instructors
• #humaninterest discussion of which students were struggling or excelling, and what to do about it
• #random mostly a place for blowing off steam
• #repo an automated channel with update messages triggered by changes in the Github repo where the course materials lived
• #showtime time sensitive course material distribution: precept materials, lesson plans, answer sheets
• #materials a private discussion channel for the course head and subset of course personnel that were writing new materials

In addition, we used direct messages a lot. Maybe 60% of traffic took place in these. The natural sub-networks were: instructor-preceptors, instructor-preceptors-tutors, and instructor-course head, with occasional bilateral conversations.

There were two face to face meetings each week. The first was a lunch meeting for all course personnel to give the course head and administrators an overview of how things were going.

For the second meeting, I organized a ‘debriefing session’ without the course head or administrators on Friday afternoon in a nearby bar. And I covered the drinks. The 'debriefing' part of the description was intended ironically but in fact turned out to be quite an accurate description of the conversations that occurred.

What worked

On Slack, automated channels were helpful, especially #calendar as a reminder that something had changed there. This meant we only need to consult this second information source when something changed.

#general was a good way to make announcements, at least after everyone had been socialized to check or be notified by Slack at regular intervals.

#humaninterest was a good way to store all discussion of students in one place. It was particularly nice to be able to look back at how we dealt with a particular student and how they responded over the course.

I used #showtime to post materials shortly before classes and precepts, usually in the form of zip files of instructor materials. I think this worked well, although it did rely on having course materials in a standardized shape in the Github repo. More on that below.

We didn't end up using #dataissues much. The problems that turned up were better dealt with offline in tutorial sessions. In hindsight I suppose I should have guessed that’s how it would work out; course infrastructure is all 'meta' stuff - course structure, whereas student data issues are course content.

Meetings Friday debriefing was unexpectedly useful for making sure course personnel were on the same page and could share information about student progress and problems.

What didn't work

Communications In addition to channels defined in terms of topic we should have had channels determined by membership. That way a lot of useful conversations would have also been archived. Also, if I understand Slack at present, only channel content can be exported.

I am not usually a fan detailed permissions setups, but we could have used the cross-cutting classification possibilities of channel structure a bit better.

Speaking of 'permissions': #repo and #materials were not terribly interesting to anyone except the instructor and course head. Although it seemed to me a worthy goal to make the course material construction process transparent to everyone, it became clear early on that the intersection of those interested in how our sausage was made and those who understood how to use our sausage-making machine (rmarkdown, R, git and github) was fairly small.

Finally, #calendar only updated when I made a new appointment to the right calendar the first time, not when it was edited, so I found myself manually writing notes on that channel to remind people that locations had changed for events. There's probably not really any way around that. Arguably those notes belonged on #general.

The course head saw all and participated in some of the Slack traffic, so it’s not clear what was gained from the face-to-face meeting with course personnel and administrators.

What we’ll change

Communications I’ll privatize some channels and make more membership rather than topic-based channels. Otherwise, it worked great.

Next time I’ll add the administrators to selected Slack channels and see if we can’t make better or different use of the face-to-face meeting time.

I've more to say about course content and its structure, but that will wait for another post.