Why SCORM 2004 failed & what that means for Tin Can

“SCORM 2004 is dying (if not already dead!).” Now that might seem like a strong statement but it’s the sad truth. For the careful observer there are many signs to support this view, and here are a few of them:

Sign #1: 75% of packages are still on SCORM 1.2, 10 years after the initial release of SCORM 2004 [1] [2]

 

Sign #2: There is no certification process for tools and packages for the latest SCORM 2004 4th edition. This is the case although several years have passed since 4th release. Currently, someone can be a 4th edition adopter but *not* certified. [3]

Sign #3: ADL itself heavily supports Tin Can as the successor of SCORM.[4]

In essence, SCORM 2004 always lived in the shadow of SCORM 1.2. Now, with the introduction of Tin Can API it seems certain that its adoption rate will decline even further.

Reasons SCORM 2004 Failed

There are a multitude of reasons why SCORM 2004 failed. Here are most prominent (and yes, we refer to SCORM 2004 in the past tense quite deliberately):

Complexity

The major contribution of SCORM 2004 was the “simple sequencing model”. In fact, it was anything but simple. It was a lot of work for LMS vendors to implement and more importantly, it was too complex for many courseware developers to use. Even the simplest of sequencing required a room full of flow-chart diagrams, dozens of field settings – and even then you needed to be an expert to actually understand what it was doing.

The sad fact is that SCORM 2004 had some nice extensions over SCORM 1.2 which generally made sense, but was hidden under the sequential model nightmare.

For example, a major problem with SCORM 1.2 was that when you took a SCORM quiz there was no way for the LMS to know what the actual questions were. You could access the kind of the question, the correct response, the student response and the score – but not the actual question. This is one of the areas where SCORM 2004 was profoundly better than SCORM 1.2. It included a full text question description and a descriptive identifier for answers. This meant that you could do some effective reporting on questions and the distribution of answers. It was a dramatic improvement but only a few took notice.

Low adoption

This was a side-effect of high complexity. Pedagogically, SCORM 2004 offered important new opportunities but at a disproportional cost.  In other words, the added benefits from the standard were unbalanced by its complexity. The end result was low adoption from vendors and instructional designers.

Even when vendors offered support for SCORM 2004 this was handicapped to great extent. For example, many rapid elearning tools that are available for creating courses DO NOT allow you to build anything easily other than basic SCORM 2004. Almost none of them have an interface for creating a dynamically sequenced Multi-SCO package.

Technology shift

10 years is a long time. Since SCORM 2004 was introduced new technologies have come and gone, smartphones have become mainstream, gamification has been introduced, Cloud & lean solutions are hot topics. We are living in a much different, and more connected, world yet SCORM is still an isolated, browser-based, LMS-centered standard. SCORM 2004 had to change in order to adapt to such a dramatically different environment and rather than do that ADL decided to save itself the trouble and start from scratch through what we now know as Tin Can or Experience API (xAPI)

On not being pragmatic

When SCORM was first introduced it answered a real-world problem – that of the standardization of learning packaging and delivery. And it succeeded because until that point in time there was no adequate way to do that job. On the other hand SCORM 2004 tried to address less obvious problems. It had a higher vision and tried to allow or enforce sound pedagogical concepts but was proven less pragmatic.

There is one important real-world issue that SCORM 2004 deliberately avoided dealing with and that is making SCORM a concrete standard. SCORM is a reference model and not a true standard – you can’t plug this into a wall and have everyone work the same way. There is still too much variation in how compliant LMSs implement UIs associated with the SCORM engine. Will content be loaded in a new window? A frameset? How large a window? How will the table of contents be presented? What navigation request does closing the browser imply? Content authors should be able to rely on a consistent set of UI expectations.

Lessons to be learned and the Tin Can future

Tin Can is trying to succeed where SCORM 2004 failed. Nothing is perfect though as ADL admits [5]. Ongoing compromises are not a bad thing per se, but can certainly be tricky.[6]

Simplicity matters

It seems like the need for simplicity is something that Tin Can endorses. Simplicity drives adoption and without adoption no standard can succeed. In essence Tin Can is much simpler that SCORM 2004 and even simpler than SCORM 1.2 (still, on the latest 0.95 and upcoming version 1 of the standard some complexity elements emerge like support for multiple languages – in essence this is good but comes with higher complexity for technology providers).

Technology-shifts can render you irrelevant

Another important characteristic of Tin Can is that it is actually technologically ‘agnostic’. It can be used inside the LMS, outside the LMS, embedded in a mobile phone or in a videogame. That provides some assurance against technology-shifts and opens new possibilities for capturing interesting learning interactions from informal activities.

Ongoing project support is important

An interesting decision regarding Tin Can is that ADL hired a company to drive the Tin Can project (Rustici Software). What that means for the future of the standard is still not clear, however, currently, the marketing and support effort is much improved.

Freedom and Standardization are opposite forces

Unfortunately, Tin Can does not help in the path towards standardization. It does however offer even more freedom to content creators by letting them, for example, define their own verbs used on statements. Interoperability of content between LMSs is somewhat improved due to the simpler messaging system and absence of Javascript; however, standardization of presentation will not be benefited from Tin Can as it is shaped today.

Tin Can chose freedom over standardization. It remains to be seen if this was a good move.

Reporting is critical for eLearning

The need for reporting is one of the main driving forces behind eLearning. Without reporting you cannot calculate the ROI (return-on-investment) of your learning activities. Reporting was not a favorite topic of SCORM but is at the core of Tin Can.  In principal, Tin Can is built around descriptors of actions (‘training evidences’) that can be translated to better reports. Still, the reporting itself depends on each vendor’s interpretation.  Also, for reporting to be useful it may help to merge statements to form higher level descriptors. For example, Tin Can can report on what you experienced or completed. But those are low level statements that cannot be rendered easily to something like “George is good at mathematics”.

Summing up

To say that SCORM 2004 failed because it was too complex is an over-simplification. There were a number of forces that led to this outcome. Tin Can tries to succeed where SCORM 2004 failed by addressing several but not all of the ongoing issues. It also comes with a fresh view on the technology landscape.

It seems that the compromises were calculated ones in order to simplify the standard but we anticipate that in the near future Tin Can will introduce several new elements in favor of standardization. Hopefully during this process its simplicity won’t be hammered too much. It is still early but a good way to introduce standardization might be through a new standard that builds over Tin Can (and thus, does not make it more complex) and addresses visual and reporting concerns. Let’s call it “Tin-Can X”.


Resources

  1. http://scorm.com/blog/2011/08/scorm-stats-then-and-now/
  2. http://scorm.com/scorm-stats/
  3. http://www.adlnet.gov/scorm/scorm-certification
  4. http://www.adlnet.gov/the-definite-indefinite-future-of-scorm
  5. http://scorm.com/project-tin-can-phase-3-known-weaknesses/
  6. http://dspace.dial.pipex.com/town/street/pl38/comp.htm

 

  • http://www.tagenata.com Amador

    Great post, Athanasios. I agree with you and now we have Tin Cap 1.0!

  • http://Www.edtechnow.net Crispin Weston

    Nice post and I agree with your conclusions. But remember that the main reason that the main reason that TinCan is simpler than SCORM is that it does very much less. A lack of emphasis on interoperability (if that turns out to be a long-term feature) will be a major issue, given that you identify lack of standardisation as one of the reasons that SCORM died.

    So I would express your conclusions a little differently. TinCan / xAPI is more of a germ of an idea than a fully sorted spec. We need to see how it develops. And 2. The problem with SCORM was not so much complexity as inflexibility. You had many different elements all hard-wired together, including at the heart of the whole thing, a data model that was originally worked out on the back of an envelope in the early 1990s. The key to the next generation will be decoupled specs that incorporate good mechanisms for extension an deprecation : ie evolutionary change.

    • Athanasios Papagelis

      Indeed TinCan can be easily seen as simplistic rather than simple. Many of its potential benefits are direct results of its simple nature (e.g, offline sychronization) rather a core part of the standard.

      I see this turn to simplicity as a good thing though. As noted on the post we can use it as the foundation on top of which other things can be developed (the same way that the 7 osi network layers build on each other).

      In essence this reflects your view as well on decoupled specs that can be extended or deprecated.

      So, I still believe that we gonna see standards that build on top of tincan and extend it in various ways that are useful for various groups of elearning professionals. It might look tempting to extend tin-can directly to integrate all those elements but I would strongly argue against it.

      • http://www.EdTechNow.net Crispin Weston

        Hi Athanosius,

        We are clearly thinking the same way here. My only question (and this is not a criticism of Tin Can but rather highlighting a paradox) is how do we strike the right balance between flexibility and innovation on one hand, and standardised interoperability on the other? If everyone uses Tin Can as one piece in a different overall mix, then they lose interoperability. This issue occurs not at the level of the individual specification, but at the level of the overall ecosystem.

        The answer, as I see it, is partly in the use of appropriate, transparent schema / service description languages; partly at the level of getting the right market pressures and incentives, so that the major players at least *want* to share data.

        Best, Crispin.

        • Athanasios Papagelis

          Hello Crispin,

          Finding the right balance is always a hard, and occasionally a futile, excercise. Being pragmatic is often the right path to follow.

          I find interoperability as the main problem of elearning standards at the moment. I would like to have a tincan object that looks and works the same way across different lms’ – even if this reduces flexibility. This does not happen today and did not happen for scorm as well. There is too much room for free interpretation from lms vendors.

          This is a problematic point even for the end-user.

          In that sense I really love the way apple gives guidelines for the interface for their IOS – take a look for example at the following url: http://developer.apple.com/library/ios/#documentation/userexperience/conceptual/mobilehig/Introduction/Introduction.html#//apple_ref/doc/uid/TP40006556-CH1-SW1

          I don’t understand why something like this cannot be enforced on TinCan as well at a second layer (what I referred to as Tin-Can X).

          • http://www.EdTechNow.net Crispin Weston

            Athanosios, Again, I think we agree that it is not so much the spec as the surrounding context. Guidelines would be good. I would add commercial incentives to ensure goodwill (certification/badging or, in the case of Apple, listing in app-store catalogues etc).

            The case of Apple is instructive, I think. Apple has a monopoly and used that to build an interoperable market. Microsoft did the same thing for desktop applications. But in both cases, it came with an anti-competitive downside at the OS level.

            For education, governments have an overwhelming interest in playing this same market-building function – but they don’t, generally through a combination of technical incompetence, lack of vision, and the self-interest of public sector bodies who are often hostile to free markets.

            Crispin.

  • Antonia

    I think you’re mistaken if you think that Tin Can will succeed where SCORM has failed.

    To develop a SCORM compliant package does not take a great deal of effort. In fact, if a SCORM JavaScript wrapper is well written then a package developer needs to know very little about SCORM.

    Currently the verboseness of xAPI JSON statements will mean that package developers will need to learn to write complicated statements and to rely on their packages to send and receive statements and messages.

    Multiple levels of difficulty for what benefit?

    • Athanasios Papagelis

      From our experience it was much faster to implement tin-can than it was for SCORM 2004 (an order of magnitutude actually) and even faster than SCORM 1.2

      The problem with Javascript is that it works only with the browser where JSON can be used in a variety of environments. And to be frank, JSON is fairly simple to use.

      Still, only time will tell if tin-can will succeed and how people will use it. I guess that by this time next year we will know :)

      • Antonia

        My apologies. I didn’t see the post when I returned to the page and I wrongly thought it had been deleted.

        • Antonia

          Oops, I meant to reply to my other post.

          Anyhow I find your comment that xAPI is an order of magnitude faster than SCORM 2004 puzzling. While it might run faster on a server I can only see lots of hair pulling for developers that try to implement it into custom packages they are building. Instead of just sending a string and a data model element they’ll need to construct some multi-element JSON with items they’ve never concerned themselves with before (what verbs can I use, what’s an activity ID, what valid URLs can I use, what does the LRS want, where do I upload a package to).

          I checked out a 3rd party product and it uses cookies to manage and, I guess, to maintain state. Cookies are a sure sign of a potential security failure.

          And that the v.1.0.0 of the specifications has numerous inconsistencies and ambiguities (as well as errors) certainly won’t help.

          • Athanasios Papagelis

            I was referring to the implementation speed from an LMS vendor point of view rather than execution speed.

            TinCan is a paradigm shift, and as always on those cases nothing works as it should at the beginning (indeed there are several incosistencies at the moment as you mention). Resistant to change is also a ongoing force here.

            However, despite those shortcommings I found TinCan much easier and practical than SCORM 2004 to implement. And the shift to JSON is something I welcome as well.

            Some constructive critisism here – thanks for sharing your view :)

          • Andrew

            Here’s another point. If I create, say, a SCORM 2004 compliant SCO then I can usually move it to *any* SCORM 2004 compliant LMS and it will run as intended – capturing all the fine granular data the SCO sends for capture. With a TC compliant course I need to either know what the particular LRS supports, recode on the server to accept what the course sends, or I need to rebuild the course accordingly. The ability to build once and then publish anywhere is lost.

  • Antonia

    I’m shocked that you deleted my post disagreeing that SCORM has failed when Tin Can has only just managed to get released (a “flawed” version 1.0.0) and had to change its name to xAPI because no one could take it seriously.

    • Athanasios Papagelis

      Hello Antonia,

      Your comment was not deleted. All comments need to be approved to be viewable.