Tag Archives: migration

Hybrid: State of the Union

It has been a while since I have written about the Sakai Hybrid integration between CLE and OAE. With some visioning help from David Goodrum and some technical wizardry by Steve Githens, we managed to hit a major milestone leading up to the Sakai 2011 Conference. This milestone allowed us to perform a live demonstration of a couple of important use cases:

  1. Using the Sakai OAE portal to launch directly into a CLE site experience.
  2. Being able to pull individual CLE tools into OAE content pages as widgets.

The first use case is important if you want a single URL for your users to enter their LMS experience regardless of whether they are primarily CLE or OAE users. This mode will allow CLE users to ease into an OAE experience while maintaining a very familiar CLE user interface.  In other words, think of it as a new portal for CLE that has some new capabilities.

The second use case is targeted more at users who are adopting the complete OAE user experience, but still need access to some functionality provided by the CLE (think of Assignments, Gradebook, and Tests & Quizzes as examples). This capability allows one to pull one or more CLE tools into an OAE content page as widgets. The other interesting thing is that the underlying technology behind this tool integration is what is known as IMS Learning Tools Interoperability or LTI for short.  This technology will not only allow OAE to pull in external CLE tools, it will also allow OAE to pull in other LTI enabled tools or applications (think EtherPad, WordPress, Moodle, or Blackboard as examples).

David Goodrum introduces the Hybrid concept and value proposition:

Lance Speelmon demonstrates Hybrid use cases:

Steve Githens expands on possibilities:

Finally we wrap up with some Q&A:

1 Comment

Filed under Education, Sakai, Technology

Sakai 3 Basic LTI… Now with more policies!

In an upcoming screencast, I will demonstrate a new Sakai 3 widget that hides all of the complexities of LTI settings from the end user. In the previous demonstration [1] you were able to see individual Sakai 2 tools living side-by-side with Sakai 3 widgets. You also saw a “sleight of hand” where I tried to paste in a 1) LTI URL, 2) LTI key, and 3) LTI shared secret without you noticing. While this made for decent “demo-ware”, it was an unreasonable set of expectations for a user who say simply wants to place a Sakai 2 Assignments tool onto a Sakai 3 page. My goal was to allow users to be able to place a Sakai 2 tool onto the page without having to know any settings.

To reach this next level of simplification, the required LTI settings would need to be supplied via administrative policies. Luckily, when I started developing the LTI service in Sakai 3, I anticipated such a need and built in some preliminary support for policies. Let’s start by taking a look at a policy, sakai.assignments.grades:

{
    "ltiurl":"http://localhost/imsblti/provider/sakai.assignment.grades",
    "ltiurl_lock":true,
    "ltikey_lock":true,
    "ltisecret_lock":true,
    "release_names":true,
    "release_names_lock":true,
    "frame_height":"100",
    "release_email":true,
    "release_email_lock":true,
    "release_principal_name":true,
    "release_principal_name_lock":true,
    "debug":false,
    "ltiKeys": {
      "ltikey":"12345",
      "ltisecret":"secret"
    }
}

All of the Basic LTI widget settings can be configured via such a policy. There are two aspects to each setting, 1) the setting itself and 2) a lock. The interaction between the two works as follows:

  1. If no setting is configured, the Basic LTI widget will not be affected in any way.
  2. If a setting is supplied (i.e. with no lock) it will become a default value for the setting in the widget.
  3. If both the setting and the corresponding lock are set, then it becomes a policy which the end user cannot change.

Hopefully this is fairly straightforward. However, we have not talked about the key to a policy itself. That is to ask, what causes an associated policy to be read and applied during the launch of the Basic LTI widget? It is another setting on the widget itself: lti_virtual_tool_id. For example, setting lti_virtual_tool_id to a value of sakai.assignments.grades would cause the policy named /var/basiclti/sakai.assignments.grades to be applied. This will allow system implementers to create many virtual tools and policies for end users, either because they want to simplify their lives with some good default values, or they need to lock some of the settings down so that end users cannot modify them. Now before you get too excited, the bad news is that while all of this logic exists and has been tested in the backend service itself, it has not been added to the Basic LTI widget yet. You will be able to see these policies in action in an upcoming blog posting where I will demonstrate a new widget called “Sakai 2 Tools”. This new widget essentially has no settings as relies completely on the new administrative policies.

While we are on the subject of policies, there is another new feature in the Basic LTI service worth mentioning. This is a little abstract, but essentially LTI has a property context_id which has meaning similar to what Sakai might call a siteId. The Sakai 3 LTI service now has the ability to manually specify the context_id if needed. This will be important to the Sakai 2/3 Hybrid integration as we will want Sakai 3 to pass the siteId of the corresponding Sakai 2 site as the context_id. Let me explain in more detail:

  1. The Sakai 2 LTI provider normally operates in the following provisioning manner:
    1. If the user does not exist, create it.
    2. If the site (i.e. context_id) does not exist, create it.
    3. If the tool placement does not exist, create it.
  2. This is all well and good if you are integrating some untrusted third party system where you want all of the LTI stuff to be completely separate from your local stuff.
  3. But if you are integrating with something closer to home (i.e. more trusted), then you probably don’t want all of this provisioning occurring.
  4. For a given LTI key, the Sakai 2 LTI provider can be placed into a trusted mode where it will not perform any account, site, or tool provisioning.

It is this trusted mode of the Sakai 2 LTI provider that we are targeting with the ability to manually specify the LTI context_id in Sakai 3. To invoke this new capability, simply set the property lti_context_id on the Sakai 3 site node to the siteId of the Sakai 2 site. As you can see from the DefaultContextIdResolver, the value of lti_context_id will be used if it exists, otherwise, the fully path to the site node will be used as the context_id.

  public String resolveContextId(final Node node) throws RepositoryException {
    LOG.debug("resolveContextId(Node {})", node);
    if (node == null) {
      throw new IllegalArgumentException("Node cannot be null");
    }

    String contextId = null;
    if (node.hasProperty(key)) {
      // we have a special context_id we can use
      contextId = node.getProperty(key).getString();
    } else {
      // just use the path
      contextId = node.getPath();
    }
    return contextId;
  }

Until next time… L

[1] https://lancespeelmon.wordpress.com/2010/04/27/mix-and-match-sakai-2-and-sakai-3-tools/

Leave a comment

Filed under Sakai, Technology

Mix and match Sakai 2 and Sakai 3 tools

I am very pleased to be able to share an important update with the Sakai community regarding the ongoing efforts to create a flexible migration path from Sakai 2 to Sakai 3. I have been blogging recently [1] [2] about the development of a new Basic LTI widget for Sakai 3. This effort builds on the existing capabilities [3] of Sakai 3 which allows you to expose entire Sakai 2 sites within the Sakai 3 portal (i.e. with the addition of Sakai 3’s social networking features). The next step in this hybrid evolution was to add the capability to expose individual Sakai 2 tools within Sakai 3 sites. Furthermore, since we were developing this new capability as a Sakai 3 widget, this means that an end user could place multiple Sakai 2 tools onto a single Sakai 3 page (i.e. something not currently possible in Sakai 2). This may sound like a subtle distinction, but as you will see in the embedded video it is an important distinction as it helps to break Sakai 2 tools out of their silos. Instructors can now author pages with custom workflows that may include both Sakai 2 tools and Sakai 3 widgets!

Skip to 12:30 into the video if you want to skip the summary and background…

So now we are beginning to see a migration path for existing Sakai 2 implementations that looks like:

  1. Sakai 2 for course sites and Sakai 3 for project sites.
    1. Reduced risk deployment as there is no hard dependency on Sakai 3 being available. For example, if Sakai 3 were to go down, users could easily be directed back to the Sakai 2 portal.
    2. Sakai 3 portal may or may not be the default portal (i.e. implementers choice).
  2. Placement of Sakai 2 tools alongside Sakai 3 pages within a Sakai 3 site.
    1. Maintains more of a Sakai 2 user experience while allowing access to Sakai 3 capabilities within a single site.
    2. Provides a nice transition between #1 and #3 for those that find it valuable.
    3. Increases risk as now users are dependent on Sakai 3 remaining available.
  3. Mixing and matching Sakai 2 and Sakai 3 tools all on the same Sakai 3 page within a Sakai 3 site.
    1. Provides the greatest flexibility for course and project site design.
    2. Provides the most “native” Sakai 3 user experience while maintaining access to Sakai 2 functionality.
    3. Custom workflows may be created with both Sakai 2 tools and Sakai 3 widgets (i.e. even if you are just exposing Sakai 2 tools, this capability does not exist in Sakai 2).
    4. Maintains same risk profile as #2.
  4. Implementer’s selection…
    1. You do not need to think of the previous three deployment strategies as mutually exclusive or synchronous.
    2. You can make any or all of these solutions available to select user populations as you see fit at any time – all based on your local goals and comfort levels.

For example, one possible migration path may look like:

  1. Build confidence in Sakai 3 and keep risk at a minimal levels.
    1. Deploy Sakai 3 alongside Sakai 2 with hybrid mode enabled.
    2. Continue to use Sakai 2 for course sites.
    3. Begin to experiment with Sakai 3 portal and project sites.
    4. Make Sakai 3 portal available to select user populations to validate access to Sakai 3 social networking capabilities as well as access to Sakai 2 sites.
    5. Make Sakai 3 project sites available to select user populations.
  2. Make Sakai 3 portal the default portal for all users.
    1. Continue to use Sakai 2 for course sites.
    2. Expand use of Sakai 3 project sites.
    3. Develop fallback plan to mitigate dependency on Sakai 3 portal availability (i.e. you still have a fully functional Sakai 2 portal sitting there).
  3. Early adopters begin relying on Sakai 3 for course work.
  4. Full adoption of Sakai 3 for both course related and collaborative work.

There is a lot to consider here and to complicate matters further I predict not one migration will be identical to another. However, we hope that we have provided enough capabilities and flexibility to tailor each migration to fit the institution’s needs. Furthermore, there is a lot experience in our community in this area as almost all of us have made a similar migration in the past. This time things are a little different… This time we have direct control over both systems and can make the migration smoother.

Let’s continue the discussion in person at the annual conference. Christian Vuerings and I will be presenting on this topic in sessions titled “Integrating Sakai 2 and Sakai 3”. There will be two regular conference sessions and a technical demonstration. See you there! L

[1] http://bit.ly/9Brbdl
[2] http://bit.ly/bXVPVI
[3] http://bit.ly/bQKjlm

2 Comments

Filed under Education, Sakai, Technology

Sakai 2->3 Integration Status Update

David Goodrum asked me to provide an update to the Oncourse Priorities Committee [1] regarding the progress made towards Sakai 2->3 integration and migration. In doing so, it became clear that this kind of update might also be useful to the community at large. There are two overarching goals to achieve in this project: 1) seamlessly exposing Sakai 2 functionality within Sakai 3 (which we are dubbing “hybrid” mode) and 2) content migration.

The hybrid mode will allow institutions to adopt Sakai 3 at a pace that is comfortable for them. For example, Sakai 3 will not have all of the functionality available in Sakai 2 day one, but their will be some compelling functionality that institutions will want to adopt before all of the functionality gaps have been closed. Hybrid mode will facilitate this approach by allowing institutions to deploy Sakai 3 and take advantage of the new features while still relying on Sakai 2 for the features not yet available in Sakai 3. The following video demonstrates the first step in realizing hybrid mode: the ability to expose entire Sakai2 sites within the Sakai 3 portal.

We anticipate that institutions will want to start using Sakai 3 for full-fledged project and committee work before course work. This capability supports this use as Sakai 3 sites can be used for project sites while Sakai 2 continues to serve the course site needs. The next step in the hybrid mode evolution will be the ability to place individual Sakai 2 tools onto Sakai 3 pages which live in Sakai 3 sites. This will give the user the ability to mix and match both Sakai 2 and Sakai 3 functionality in a single user experience.

Next: content migration. The primary use case that we are tackling first is the ability of an instructor who taught a class last semester in Sakai 2 to be able to import that site template into Sakai 3. The following video demonstrates an initial deliverable where one can export content from a Sakai 2 site and then import that content into a Sakai 3 site. Please keep in mind that this is very early work and is rough around the edges, but the methodology and underlying service is sound.

Moving forward, focus in the following areas is required:

  1. Exposing individual Sakai 2 tools on Sakai 3 pages within Sakai 3 sites.
  2. Exploring a results mechanism so that a tool hosted in Sakai 2 can report results to a tool hosted in Sakai 3 (and vice-versa). As an example, you might want to use a Sakai 3 Assignments tool which will need to report outcomes to a Sakai 2 Gradebook.
  3. Continued effort to import Sakai 2 site templates into Sakai 3 for semester-to-semester transitions.
  4. Deeper content migration that will not only transfer site templates, but also include full user content. For example, this will likely be a requirement to migrate a project site from Sakai 2->3.

3 Comments

Filed under Sakai

Importing content from Sakai 2 into Sakai 3 (take 1)

Development was starting to slow down for me on the Sakai2+3 Hybrid Mode, so I needed to turn my primary focus elsewhere. Michael Korcuska and I had decided previously that the next focus point would be to develop a working prototype that would allow someone to take a zip file exported from Sakai 2’s Site Archive tool and import the content into Sakai 3. Initially the scope would be limited to just the content contained within the Resources tool (a.k.a. ContentHostingService) since Sakai 3 currently has enough functionality to support the files and folders model.

When I started down this path, I did not expect to reach a stopping point by the end of the week. Frankly I thought it would take longer. But after a couple of days, I had the logic around parsing the content.xml file and extracting the content into my local file system working pretty well. The next couple of days were spent porting this working code into Kernel2 as a SlingServlet and creating a RESTful web service. After a couple of bumps in the road and someone moving my cheese, I am pleased to say that the first iteration of this work is complete.

As an example, you can take the sample archive.zip file which came from a Sakai 2 test instance, and upload it to Sakai 3:

curl -F"path=/site/import/folder" -F"Filedata=@archive.zip" http://username:password@localhost:8080/foo.sitearchive.json

The web service expects two parameters:

  1. path: The path to a folder where you want the content imported.
  2. Filedata: one or more zip files to import.

The result will be a folder that looks like the following screen shot:

While there is still much to be done (e.g. mapping file meta-data, support for more resource types, etc.), this is an important first step. For one, it demonstrates technical feasibility. Secondly, it creates the beginnings of a framework that can be extended to support importing other Sakai 2 tools, and eventually other import formats entirely like IMS Common Cartridge. If you are interested in looking at the code, it can be found at my github repository.

Looking forward, I will likely begin investigating IMS Basic LTI as a mechanism to enhance the Sakai 2+3 Hybrid capabilities. Currently, the hybrid mode supports entire sites (i.e. the user chooses to enter either a Sakai 3 site or a Sakai 2 site via the Sakai 3 portal). Ideally, one should be able to mix and match tools from either Sakai 2 or 3 in a Sakai 3 site. Dr. Chuck has done some good work in this area – Sakai 2.7.0 will have both a BasicLTI consumer and producer. So theoretically, if Sakai 3 had a BasicLTI consumer, it could present a Sakai 2 tool to a user as a Sakai 3 widget. My hopes are that among Dr. Chuck, Noah Botimer, and myself that we could turn out a Sakai 3 LTI consumer relatively quickly. More to come in the new year. Best regards, L

7 Comments

Filed under Java, Sakai

Investigating site exports from Sakai 2

So the export/import file format investigation has reached some early conclusions regarding the use of Moodle’s backup schema and it looks like we will be looking elsewhere. See: Moodle export-import format investigation and the email thread itself.

While we ponder IMS Commmon Cartridge, I thought I would investigate what it would take to provide the capability of exporting Sakai 2 sites into the existing Sakai 2 proprietary XML format. This is a long standing request within the Sakai community, but one that no one has been willing to tackle. This is a bit of a dodgy situation as most tools do participate in the method EntityTransferrer.transferCopyEntities(), so it is possible to copy the structure of a site from semester to semester. I use the term “structure” because this is common practice among LMS applications to only copy what might be termed a “template” across semesters. For example, this copy process would include content like forum definitions, but not student responses; grade book items, but not student grades, etc.  The primary use case is an instructor who taught a class last semester can import that previous site into the current semester’s course site to reduce setup time.

So far so good – but here is where things get a bit dodgy… The EntityTransferrer.transferCopyEntities() method copies entities directly from one site to another (i.e. without writing any of these entities to XML). While Sakai 2 does have a mechanism for writing entities to XML, called ArchiveService.archive(),there are at least two problems with it: 1) Unlike transferCopyEntities(), all student positngs, grades, etc. are included in the XML produced (i.e. more like  a site backup), and 2) only a small subset of tools actually implement the ArchiveService.archive() interface! So this leaves me wondering:

  1. Does anyone actually depend on ArchiveService.archive()? My instincts tell me no since most of the tools do not implement it. Am I wrong?
  2. Could we usurp the ArchiveService.archive() interface and change the behavior so that only site structure is exported without student content?
  3. Do we leave ArchiveService.archive() alone and create a new API?
  4. How many tools still need to implement archive()?


1 Comment

Filed under Java, Sakai

Sakai 3 export/import formats research

Since Sakai 3 is a ground-up rewrite, there is plenty of opportunities to rethink assumptions that have accumulated over the years.  One of those areas where a fresh look should do some good is in course export/import formats.  Sakai 2 has its own proprietary format which provides a “full fidelity” capability to move from one course site to another without losing anything.  While not losing anything is desirable, it comes at quite a cost; i.e. not being compatible with anything else on the planet.  This approach is not uncommon and I see now that Moodle also has its own proprietary format.

While there are some good open standards for course export/import (e.g. IMS Common Cartridge or SCORM), they will not provide a “full fidelity” export/import workflow where one could export from Sakai 2 and then import into Sakai 3; i.e. some information would get lost in the translation.  Now, supporting these open standards would have other important benefits, they are not first order solutions for simply getting from Sakai 2->3.  My hopes are that one day an open standard could be expressive enough to cover such a use case, but the innovation curve in the applications and tools may always exceed the ability of a lowest common denominator solution.

With that said, we thought it would be beneficial to see if Moodle’s export/import format provided enough capability to move data from Sakai 2->3 without losing any critical structure.  If this worked out, we would have one great feature out-of-the-box: the ability to move from a course from Moodle to Sakai and vice-versa.  It will be interesting to see how this works out and I will keep you updated as progress is made…

Leave a comment

Filed under Education, Sakai, Technology