User Experience, Design Choices, and Learning Analytics: A lesson learned with ProSolo

Design choices made in different phases of the development of a socio-technical system may have implications on user experience that are not easily anticipated. Our intentions have always been to provide the users of ProSolo with the best possible experience. However, one such design decision turned out to require further consideration after a DALMOOC participant reported an issue with the automatic logout from ProSolo. Rightfully, the user wanted better experience as being logged out automatically can be annoying. This was caused by our decision to set the time of automatic logout after 10 minutes of inactivity. The reasons for us making this decision were multi-fold, and here we touch on the three major ones – security of user data, use of servers and associated environmental impact, and accuracy of learning analytics. With this, we would like to reach out to the ProSolo users and ask for their input how to approach this issue and balance these three factors.

Being logged in to any Web application when losing either a laptop or mobile phone can pose a serious threat to the security of user data. Not to mention the potential abuse of the user identity! The user just needs that inconvenience in addition of the loss of their favorite device. Similarly, the use of a personal computer creates a habit that many don’t logout from any application. Why would we, as that’s another thing to type in? But, forgetting to logout, while working at a public computer (e.g. lab), can again create a similar data security threat. In all these situations, it is ultimately the user who personally gets affected with the decision to have a long timeout (or no timeout at) without affecting  others with that decision. And, who is the software development group to make that choice on behalf of the user, isn’t?!  Thus, from the software development, this case is easiest to deal with and simply to follow fully the user choice whatever that be. Is that ideal?

Cost and environmental implications are connected with an increased use of servers if users are not automatically logged out after some period of inactivity. If a user is not active, or they have forgotten to close a window in which the application was open, the servers will still be allocated for that user. While it might not be visible, any application such as ProSolo has a distributed server architecture. If one server gets too busy when serving several users, a new server needs to be initiated (through the process known as auto-scaling) in order to serve all the users. If users stay logged in, we need more servers and that increases the cost of hosting. For the volume of the use during DALMOOC and the generous support of UT Arlington for the hosting servers, this is not a major concern for ProSolo at this point. Even with the growth of the user base, the financial resources can be found. However, we need not to forget the potential environmental impact – that is, the use of additional servers means more energy spent. If the energy does not come from green sources (as common today, unfortunately), that’s not a desirable outcome we would like to see. For the moment, similar to the costs, that’s not the major issue for ProSolo given the amount of resources needed. It will soon become we hope, especially, if we want to promote environmentally friendly technology – which is our goal! In this case, the user experience may also have broader implications than just security and becomes a question how to balance the user experience needs and responsible energy consumption?

Learning analytics are less accurate if we don’t know when users become inactive and logged out. Together with Vitomir Kovanovic, Srecko Joksimovic and Ryan Baker, members of the ProSolo team Shane Dawson and Dragan Gasevic have recently submitted a paper to the LAK’15 conference. In the paper, they analyzed different problems of estimation of time users spend online in learning technologies like LMSs (btw – we are happy to share the paper draft, if there is interest). When we have a clear logout event logged with a learning tool, as originally used in ProSolo, the estimation of time becomes much more accurate. Otherwise, different strategies for estimation of time spent online are needed (in the mentioned paper, 15 strategies were analyzed). The choice of those strategies in the estimation of time on task can have a big impact on the results of learning analytics. The above study showed that the differences in the explained variability of regression models – gauging learning success based on time spent online – can be as much as 15%. That is, a regression model with one time-online estimation strategy explained about 45% in the variability of the learning outcome (the student grades), while another one only 30%. An even bigger problem is that we don’t know – without further observational studies – which estimation strategy is best (or optimal).

The above research led us to the decision to introduce the timeout of 10 minutes to increase the accuracy of the measurement of time spent actively using ProSolo. Informed by the existing research, we thought that this would not hurt user experience. As turned out, we were not quite correct! This time for automatic logout should have led to the two major benefits learning analytics research: i) we would know which estimation strategy was the most appropriate to use when/if we even remove mandatory timeout; and ii) any analytics performed based on the data collected during this course will be much more accurate. Although the benefits of more accurate analytics should improve user experience in the long run (why else working on learning analytics?!), reasons of this nature cannot easily be justified and should not be given a priority over the user experience.  Alternative study designs for time estimation should be used instead!

We hope that this post sheds some lights on the complexity associated with the design choices when developing a software feature that seems rather simple on the surface. As the user perspective and review is the most critical driving factor for the design and future development of ProSolo, we look forward to learning about your ideas and reflections how this and related issues should be addressed.

In the following blogs, we will discuss other lessons learned and important questions raised by DALMOOC participants and ProSolo users.

Thank you!

Basic steps in ProSolo

In this post, we share some basic information how to use ProSolo through a couple of video tutorials. In the first video tutorial, we describe the basic functionality of ProSolo, how to set up a ProSolo profile and connect it with other social media you use, and how ProSolo can make user contributions visible to others and archived through daily digests.

 

In the second video, we describe the ProSolo functionality that supports learning and assessment. In particular, the video explains the main terminology that ProSolo users will encounter in their learning; that is, credentials, competences, and activities. The video also explains the entire lifecycle of competence development in ProSolo – from planning to the presentation of the completed and assessed credentials and competences in personal profiles.

 

We hope you will find these videos useful in understanding the conceptual model of learning ProSolo supports and its functionality that implements its model. In the coming days, we will share more tutorials that will offer additional insight into recommended and best practices how to use ProSolo. We look forward to seeing you in ProSolo and getting your feedback.

Thank you!
The ProSolo team

ProSolo Launched

Welcome to the ProSolo blog.

ProSolo is a social learning platform for the development and credentialing of competencies. . The concept of ProSolo was initially inspired by the results of an earlier research project with several European Partners.

The education sector is undergoing an unprecedented level of change. From open badges to course modularization to the growth of open education resources such as MOOCs are but a subset of the drivers impacting on how we undertake and provide education. In developing ProSolo, we recognized that the traditional model of education and training of classroom centered practices and formalized credentials were rapidly being challenged, in part due to the growth of new and open education models. Increasingly, private, and public industry are now recognizing the importance for all forms of learning from formal, to informal and workplace based. There is growing acknowledgement within and outside the education sector that the achievement of competencies, learning outcomes and awarding of credentials is not limited and cannot be limited to universities and colleges alone.

ProSolo harnesses the transformative power of adaptive and personalized learning to unbundle degrees, programs and courses and re-aggregate in new ways that better match each student’s learning needs, interests and career goals. ProSolo provides all learners with the ability to personalize their education experience. There has been much anticipation and debate around the capacity for technology to provide “adaptive and personalized learning”. ProSolo is developed with an intention to provide learners with the opportunity to customize, modify and personalize their learning journey. This is done with a belief that education is for everyone and learning occurs everywhere. ProSolo’s design and conceptual model captures the essence of personalized learning. ProSolo’ provides users with the ability to unbundle education programs, courses and units into discrete yet inter-related competencies to allow learners to construct their education pathway in a manner that better reflects their interests and future career motivations and requirements.

At the core of ProSolo are the following principles:

  • competence is a social object –  learners develop their competencies by harnessing the opportunities offered by social media and connections learners create while learning, working, or playing;
  • learners are unique and so are their profiles – understanding and documenting what learners know, regardless of where learning happens, is essential for creating adaptive learning opportunities;
  • learning is credentialed – although learning may occur in informal and authentic contexts, a robust credentialing pipeline is necessary to assess and validate the evidence learners provide in order to demonstrate the achievement of stated competencies.

The growth of informal learning opportunities such as Massive Open Online Courses and their associated certifications has forced universities to rethink the processes for credentialing and learning to establish more flexible models for recognizing alternate educational pathways. ProSolo’s mapping of personal goals and experiences directly to the achievement of competencies and granting of credentials provides learners with greater flexibility in their study options, as well as the recognition of alternate learning pathways and prior experience. For education providers, ProSolo maps curriculum activities directly to learner competencies and outcomes allowing for easy unbundling and reassembling of degree programs and courses. ProSolo unpacks the rigidity of existing programs to cater for alternate educational pathways and provide students with new opportunities to gain recognition for prior and alternate credentials and experiences. ProSolo doesn’t just break the concept of the credit hour – it totally removes it.

After performing numerous user experience studies, we have officially launched ProSolo Beta for the use in Data, Analytics and Learning (DALMOOC) offered in partnership with edX, University of  Texas at Arlington, Carnegie Mellon University, Teachers College, Columbia University, and Athabasca University. With its innovative course design, DALMOOC creates a great opportunity to learn more about the needs of learners for creating a learning environment that  will help them develop and document their learning experience, link them with social networks of relevance for their learning goals, and credential competencies acquired through their personal pathways.

During this stage, ProSolo can be accessed by using the accounts created with edX. Once enrolled in the DALMOOC, ProSolo will allow users to create their ProSolo accounts and continue using it without the need for any external identity management provider.

This is merely the first blog in a series about ProSolo. In future blogs, we will touch on many important concepts as well as providing tutorials such as how to use ProSolo, social learning in ProSolo, credentialing work, what the ProSolo socio-technical architecture looks like,  and what the use and implications of ProSolo are for higher education. We look forward to your feedback and comments.

Thank you!

The ProSolo team

 

 

A blog of ProSolo Technologies Inc.