Developing the TU Delft My Timetables for Blackboard application

This week a new issue of the W.I.S.V. ‘Christiaan Huygens’ MaCHazine is released. Together with Marco i wrote an article about our My Timetable project which i would like to share with you: Back in August 2009 the Delft University of Technology switched to a new application for building and maintaining schedules. This application, called Syllabus+, provided the schedule makers with an easy way of scheduling activities more efficiently across the whole campus. A new interface came with the system which is now known as roosters.tudelft.nl/timetables.tudelft.nl. During the first weeks some startup problems occurred, but when all the problems were solved the system worked fine. Unfortunately ‘worked fine’ does not mean it works the way you want it too. While O&S was happy with all the new features, students were left with an interface not suiting their needs. As chair of the faculty student council for the faculty EEMCS, I was one of the first to receive complaints from students. Students of our faculty know how to complain when the university delivers an imperfect IT system. Complaints varied from “I’m not able to get my elective courses in my schedule”, “why do I have to select everything over and over again to check for changes” to “my phone uses a digital calendar so iCalendar export should be available”. Workarounds by different students showed up within a few days, but none of them really worked perfect. All the complaints led to a commitment by the TU that a new, better interface would be available at the start of the second semester. TU Delft application developer 3xO unfortunately lacked the manpower to build a new system in time, so in October word reached us that they were looking for students to get the job done. That’s when I teamed up with Marco Krikke, Mike Noordermeer and Maarten van der Beek to build a new schedule interface from scratch, based on student needs. Constraints Before we even started working on the project, it was necessary to define the scope of the project. First of all we were required to integrate our new interface into BlackBoard. This forced us to make use of Java, and one of us had to study BlackBoard documentation, since nobody had any BlackBoard programming experience. All the data that we wanted to store also had to be saved in BlackBoard. All the timetable data was provided in a Microsoft SQL Server database. Basically our challenge was to build a Java based building block for Blackboard, which takes data from a MSSQL database and presents it to users in an easy and understandable format. Requirements The constraints did not seem to be anything more than a challenge, so we moved on to gathering requirements. Since we now had the opportunity, we were eager to fix the problem of the schedule interface once and for all. Mike and Marco had already assisted our faculty in building a list of requirements for the old Webber system, so we had something for a start. After interviews with students, O&S, 3xO, Oras and educational commissioners from the study societies, a full list of requirements was created. I will not bother you with a full list of functional requirements, but summarized our system would provide the following features: add courses to a profile based on course name, course code, study program, BlackBoard enrollments, student number (for Architecture students) and staff member (for staff members). We went to a meeting with 3xO with a complete requirements analysis document, and after some negotiating we came to an agreement. Now we could finally start with the real work. Architecture First the basic architecture of the system had to be defined. Since everything had to be done in Java, starting with Hibernate as an Object-Relational Mapper was an easy decision. The interface itself had to be clear, easy to use and highly interactive. The Google Web Toolkit (GWT) offers all the functionality to make this possible. The combination of GWT and Hibernate is the technical basis for the application. Some other libraries were also added to support common used functionality. Among this list are log4j for easy logging; ical4j, opencsv and itext for generating the different export files and some BlackBoard libraries to support the BlackBoard integration. We will discuss all the libraries in more detail later on. Database All timetable data is provided in a MSSQL database generated by a brand new module for Syllabus+. In the server layer of our application, Hibernate is used to map information from the database to usable Java-object. Due to the complexity of the database, performance proved to be an issue during development. The first basic implementation of Hibernate performed okay only for small data requests, but after some optimizing this also worked for larger data sets, such as full schedules. Performance issues returned when we implemented support for different student sets per course later on. The first time we saw a java.lang.OutOfMemoryError it came as a surprise, but after some new rounds of optimizing the whole ORM-layer performed good enough. Some tweaks to the Hibernate configuration ensure fast en reliable performance with a higher user load. User Interface From a user point of view everything has to work fast and intuitive. Nobody wants to spend time learning how to create its personal timetable. GWT does not only provide easy to use and recognizable interface objects, but also a full framework to manage the client server communication. Communication is done with asynchronous remote procedure calls to the server, possibly resulting in so called delta updates at the UI. Delta updates only update certain elements of a page, so the page does not have to be completely refreshed after each user action. Development is done in Java, but all code is compiled to JavaScript by a special GWT compiler. SmartGWT, an extension framework to ‘normal’ GWT, seemed to provide the best set of interface features for our application. After some implementation had been done everything seemed to work, but stability and fast performance proved to be difficult. Switching back to normal GWT was the only option left. This clearly improved the responsiveness of the whole interface but some of the features had to be manually implemented, such as the graphical calendar view. Implementing the whole interface took more time than expected, but in the end everything worked out fine. Since GWT compiles HTML and JavaScript for different sets of browsers to overcome rendering issues, our interface had to be tested for browser compatibility. The newest versions of all major browsers are able to handle the JavaScript without a problem, except for some strange layout bugs, but some of the older browsers proved to be a problem. Everybody knows JavaScript is not processed quickly in IE6, but getting a cup of coffee is the least you need to do when trying to run a GWT application in IE6. Due to some strange problems, GWT also recognized IE7 as IE6 when we first tested browser compatibility. Every TU Delft computer still runs IE7 so decent support was required. Fortunately some workarounds proved to be good enough to get everything going in IE7, but still IE8 is recommended. Blackboard Running a GWT application in BlackBoard did not require many adjustments. Since BlackBoard still uses frames, all you have to do is put the GWT application in its own frame. All TU Delft BlackBoard users will recognize the big top frame with the TU Delft logo. This causes no problems during normal usage, especially since the top frame is replaced by a small one when browsing course pages. The same small top frame is desirable when displaying someone’s schedule on screen to use the most amount of screen space available, but this functionality is not offered by BlackBoard. It is always big, always small or only small when browsing courses. Clearly the last option applies to the TU Delft implementation. By using a nasty JavaScript hack, we are still able to resize the top frame, but this is something you should not even want to use in a serious application. BlackBoard has some more ‘unexpected’ features. Let’s take a look at the user authentication for example. Presenting someone’s timetable is only possible when a user is logged on, so we called the BbSession.isAuthenticated() method before doing anything else. Soon enough we discovered this method returns True, even when the user did not log on yet. In that case BlackBoard sees an user ‘authenticated’ as guest in that case. Solving this issue was done getting the required information from the BlackBoard ‘context’ (ctx) by using this code: if (ctx.hasUserContext()) { User user = ctx.getUser(); User.SystemRole role = user.getSystemRole(); isLoggedIn = role != User.SystemRole.GUEST; } We came across another surprise when implementing the ‘import from BlackBoard’ feature for adding courses to your profile. Getting a list of courses a user is currently enrolled in takes more time than you would expect. Every course in Blackboard is defined as a Course object. Course and User are connected through a CourseMembership object. To get a list of active BlackBoard courses retrieving all CourseMemberships for a user should be enough. Unfortunately, this list contains all the courses a user has ever been enrolled in. Also a difference has to be made between ‘normal’ courses and ‘communities’. Putting it all together results in the following code: CourseDbLoader courseLoader = CourseDbLoader.Default.getInstance(); CourseMembershipDbLoader membershipLoader = CourseMembershipDbLoader.Default.getInstance(); List<CourseMembership> memberships = membershipLoader.loadByUserId(user.getId()); if (memberships.isEmpty()) { return new HashSet<String>(); } else { for (CourseMembership  m : memberships) { Course c = courseLoader.loadById(m.getCourseId()); if ((c.getServiceLevelType() == Course.ServiceLevel.FULL)&& c.getIsAvailable() && m.getIsAvailable()){ Export While implementing the export functionality, we also came across some unexpected problems. For example generating an iCalendar file from a list of activities does not require rocket science. Unfortunately, it does when support for both Outlook 2003 and Google Calendar is necessary and you don’t happen to be in the UK. Specifying a time zone for the iCal file does not work with Outlook 2003, but not specifying it makes Google Calendar think it is GMT instead of GMT+1. The only way to get this to work is defining a time zone per activity instead of per file. Generating decent PDF output also proved to be difficult. Using the iText library for a clean text output is no problem at all, but generating a decent graphical week view is difficult. We do know the current graphical PDF export is not perfect yet, we will improve it if the TU requests an updated version. Testing When the final release came near, testing became more important. Fortunately, a lot of students helped us testing the system to ensure it complied with all student needs. We have been able to solve a lot of the minor problems reported, but testing also brought us a list of ‘like to have’-features for possible future releases. We do not plan on abandoning the project anytime soon, so when the first weeks of usage are a success, we will definitely look into some new features. Some of these features require adjustments require action from O&S, others might be implemented in a future release. Conclusion Developing the MyTimetable application has been quite an experience. Like all IT projects, we came across some unexpected problems, but in the end everything works fine. Almost all feedback we received has been positive so far, but we also got some interesting new ideas for possible future updates. All comments and suggestions are still welcome at tom [at] eveoh [dot] nl. Want to thank us for the new system? You can find us in the /Pub every Wednesday starting at 4.