Low-Cost Test and Quiz Tool Comparison

I’ve recently been working with Sameer Bhatia the CEO of ProProfs as I was going through an evaluation of various online tools that support easy, fast, low-cost testing and quiz tools (disclosure). He helped me pull together the following evaluation of a few different solutions:

• ClassMarker
• Quia
• ProProfs
• Google Docs Forms Creator, and
• Quibblo.

Evaluation Criteria

For the specific need I was looking at, I was looking for the following out of a solution:

• Cost: Is the tool free or very-low cost to use? Trial periods wouldn’t count.
• Question Types/Methods: Does the tool offer the ability to create a rich testing experience by offering various question types?
• Multimedia: Does the tool offer the ability to include audio and/or visuals for questions?
• Data Collection: Does the tool provide detailed statistics on individual and/or group performance?
• Grading: Does the tool allow for several methods of grading responses? (i.e. immediate feedback, final results with or without which questions were missed, etc.)
• Download Test Results? : Can the test data be retrieved for analysis or backup?
• Layout/Accessibility: Can the layout of the quiz output be changed and does it handle accessibility issues?
• Security: Are there security methods in place to catch/prevent cheating, hacking, or other similar problems?
• Ease of Use: Is the tool well supported and easy to learn?

ProProfs

ProProfs Quiz Maker is an easy to use quiz program with a good variety of question types and options. It’s pretty amazing that it has 1 million users, over 100,000 quizzes in its public library, and over a million quiz takers. A nice feature is the end of quiz certificates for completed quizzes that can be customized. Also, test results can be downloaded in several formats. It is good at supporting accessibility with its one question per page to make the use of a screen reader more effective. It has a free and a low-cost version. To access unlimited grade reports, a paid subscription is required.

Best Features

• Very easy to use – I created a quiz in minutes.
• Good variety of question types.
• Good option for users with accessibility issues.
• Customizable quiz completion certificates.
• Relatively new to corporate quizzes and tests, but has some nice features for this kind of need.

Weaknesses

• Free version comes with Ten free grading reports - paid versions can have unlimited reports.

To see a side-by-side comparison, please see the table below.

ClassMarker

ClassMarker allows you to create simple tests/quizzes for free and seems well suited to classroom use. Instructors are able to modify score reports to provide feedback. For some of the more useful features, such as viewing score reports, an active paid account is required.

Best Features

• The program allows users to set up a ‘class’ so there is a unique login and password to track users.
• Final score reports can be modified to be shown as the instructor wishes.
• Tests can be paused, saved, and continued at the learner’s convenience.
• Variety of question types can make for a dynamic quiz.

Weaknesses

• All of the more useful features require a paid account, and even score reports cannot be viewed without one.
• No image/video support for questions.
• Not as easy to use as ProProfs.

To see a side-by-side comparison, please see the table below.

Quia

Quia allows all users to create many types of quizzes with a large variety of question types that can also include audio and video features. The instructor can set up various security settings for each class to provide more secure grading and decrease the risk of cheating. However, I found it a bit harder to use that the other tools.

Best Features

• Large variety of question types available.
• All quiz results are available in the free version.
• Security features designed to prevent cheating.

Weaknesses

• If students are not registered for a ‘class’, then the instructor cannot view the data from the quizzes.
• There is no way to choose how score reports are shown to students.
• Not as easy to use as some of the other tools.

To see a side-by-side comparison, please see the table below.

Google Docs Forms

Google Docs has a built-in Form document type that allows the creation of a set of questions. This can be an interesting quiz / test tool especially if you already use Google docs. There are a fair number of question types available and it’s very easy to use. However, users are anonymous while they are taking the test/quiz and thus need to self-report who they are (through a form field). This means that cheating would be quite easy. The reality is that this is more for creating surveys than creating quizzes and tests, but definitely worth considering in some situations.

Best Features

• Free and easy to use
• Data can be sent right to a Google docs spreadsheet or basic summary page
• Good variety of question types

Weaknesses

• No ‘grading’ can be done on quizzes
• No cheating deterrents
• Not designed for quizzes

Google docs is not included in table below since it is not designed for quizzes

Quibblo

Quibblo is a quiz and survey system that is a bit limited in terms of question types which makes it a bit harder to create a typical quiz or test. Instead users can create the type of question they require by manipulating the instructions and fields within Quibblo.

Best Features

• Free quiz designer

Weaknesses

• Site geared more for social settings than academic or corporate
• Lots of advertisements along the sides of quizzes- distracting

Quibblo is not included in comparison table as its not an academic or corporate quiz site.

	Classmarker	Quia	ProProfs
Cost	Free (some features require purchase) $24.95 - $49.95 per year	Basic is Free. $49 per year for Educators, $199 per year for Corporations	Personal use version is free. Educator version is $2.97 per month, Business version is $16.67 per month
Question Types/Methods	Multiple choice Punctuation Essay Multiple response True/ False Free text	Multiple choice True/False Pop-up Multiple correct Fill-in Initial answer Short answer Essay Matching Ordering	Multiple Choice Multiple response True/False Fill-in Essay Short answer
Multimedia	No image/video support for questions. Does allow for specialized characters.	Yes. Can add an image or audio URL to a question.	Yes. Supports image/video for questions.
Data Collection	Can only see score results with a paid account.	You can see the results of students in your class.	10 free reports are displayed. Premium version comes with unlimited reports.
Grading	Yes. Final score reports can be modified to be shown as the instructor wishes. Students can also save and finish the test later.	No, students will know their scores at the end of the quiz, but the instructor cannot choose how the reports are shown to students.	Yes. Several scoring options exist along with end of quiz certificate customization.
Download Test Results?	No, however, everything is backed up on the server. It cannot be downloaded.	Yes. If students are registered to the ‘class’, then the instructor can view the grades. If there is not a ‘class’ created, then there is no way to retrieve the data.	Yes, data is downloadable in several formats.
Layout Accessibility	This cannot be changed. Overall the layout can get a little hard to follow and can take some time to get used to.	This cannot be changed. The layout is easy to understand, and someone with visual difficulties should not have problems.	This cannot be changed (though it supports one question per page or all on one page option). Someone with visual difficulties should not have problems.
Security	Each user has a unique login and password, so the information is kept secure.	Access can be restricted to the ‘class’ only, and reduces the risk of cheating.	Users all receive a unique login and password to ensure security. Quizzes can also be protected by common password.
Ease of Use	Not as easy to use as some programs.	Not as easy to use as some programs. More time was spent viewing the tutorials than creating a quiz, and even then it was still somewhat confusing.	Very easy to use. Clear and concise instructions.
Overall	Lack of reporting in the free version is not great, but it’s low-cost for the version with score reports.	The variety of questions is great, as is the ability to add multimedia to a question. However, the ‘create a class’ function is potentially daunting, and it is not as easy to use as a whole as some programs.	Nice program with good features, and very easy to use. Low cost.

More about → Low-Cost Test and Quiz Tool Comparison

Sales eLearning – 21 Great Resources

I was asked about approaches for eLearning for Sales People.  I’ve had quite a bit of experience with this and actually one of my very first projects was creating a pretty incredible eLearning solutions for Lexus sales people.  Of course, it’s such a big topic that I decided to cheat and quickly point the person to eLearning Learning and particular to eLearning Sales, eLearning Sales Metrics, Sales eLearning Case Studies, and Sales Performance Support.

Instructional and Performance Strategies for Sales eLearning

1. Learning Theory 101 for Sales and Marketing Professionals (Part 1)- Business Casual, October 13, 2008

   When consulting with sales and marketing professionals who work for companies that create training products and programs, I try not to be a "training purist" when discussion goes to "positioning" their product for the market. Specifically, the web. In Part 2, I'll list some of the levels within one of the learning types.

2. Learning Theory 101 for Sales and Marketing Professionals (Part 2)- Business Casual, October 18, 2008

3. Learning Theory 101 For Sales and Marketing Professionals (Part 3-Last)- Business Casual, October 24, 2008

4. eLearning Development: Instructional Design Lessons from a Sales Trainer- Upside Learning Blog, July 11, 2009

   That’s because I’ve been in training this week; and would you believe it was Retail Sales training. What’s an instructional designer doing attending sales training? We’ve been tasked with converting their Retail Sales Training seminar into an eLearning package. Yes, I know haven’t posted anything interesting last few days.

5. Data Driven- eLearning Technology, December 10, 2008

   Will the retail sales training change behavior in ways that improve customer satisfaction? Understanding this model is important in order to be able to apply it within different situations in order to help drive behavior change that ultimately leads to improvement in metrics. Most action plans last four to six weeks.

6. A Formula for Storytelling in eLearning- Integrated Learnings, April 6, 2010

   When a client asked our team to develop a sales skills enhancement course, we opted to base the training on the methods of the most successful sales person in the department. By Shelley A. Gable. One of the things I love about instructional design is that it engages me both analytically and creatively. Illustrate a concept.

7. Learning Performance Business Talent Focus- eLearning Technology, July 15, 2009

   Performance – Responsible for ensuring that behavior change occurs in a way that improves performance. All things being equal (which they never are), I'm guessing my focus is more a business and performance focus. Certainly Work Literacy has a learning and performance focus and is a passion of mine. Does this ring true?

8. Work Context: The New Classroom- Living in Learning, March 22, 2010

   Fifteen of those years were spent living out of a suitcase as a roving sale trainer and then as a manager. It was shocking to me because I had convinced myself – after more frequent flyer miles and Marriott points than a human could use – that I had the sales training gig figured out. And I did. As oddly as it may sound.I

9. Harvesting Learning’s Fruit: A Downstream Training Investment- Living in Learning, September 4, 2009

   Learners, managers, training staff, and SMEs all have a vested interest in what happens after training with respect to reinforcing learning through coaching, knowledge sharing, and in the case of sales people – boasting and bragging. Yes, I was in sales for many years and know what to expect. Refer to Figure 3.0 Figure 3.0.

10. The Business Case for Social Learning- Daretoshare, April 13, 2009

    Job losses, combined with cutbacks in workforce development investments, mean that many companies are less able to serve customers, make sales or generate innovations than they were just a few months ago. The Business Case for Social Learning. In effect, their capabilities are in recession.

11. Creating scenarios for learning - a four-step guide- Bottom-Line Performance, May 27, 2009

    If you are creating a sales training course, for example, your challenge might be the need to increase sales in a territory. The first course was designed to teach new sales reps basic electrical terminology. Tom Kuhler and his “Rapid e-learning&# blog once again bats 1000 by sharing tips for creating e-learning scenarios.

Technical Approaches and Case Studies for Sales eLearning

1. Sales Team Portal for Content and Expert Access- eLearning Technology, October 12, 2009

   I wanted to share some discussions I've recently had around an interesting project where the company provided the sales team a collaboration, content distribution and expert access portal. Their large sales team sells a diverse and complex set of products and services to multiple markets. Do you have a case study for me? learning.

2. Shift in eLearning from Pure Courseware towards Reference Hybrids- eLearning Technology, May 25, 2006

   sales. Most of the content is presented as static web pages that tell how to perform particular tasks, but some of the pages contain embedded Captivate movies to demonstrate or simulate use of the system. Just when you've made the transition from the prior generation of CBT authoring tools (e.g., It is asynchronous.

3. Using the iPad in Sales Training: Case Study by Lora Davis- Learning Solutions Magazine, July 19, 2010

   instruction, one sales training team has already been successful in using Apple’s new device in blended. While many e-Learning producers are struggling to understand how the iPad may be useful in online. instruction in the classroom. Here’s their story.

4. Case studies of corporate (social) learning- March 12, 2010

   The Business Case for Social Learning , Accenture, Point of View, April 2009 Sun Learning eXchange " When the company�s sales division reduced training hours and budget, Sun Learning Services (SLS) needed to get strategic. Here are some examples of social learning in organisations. Im looking for the Why? and How?]

5. 10 Strategies for Integrating Learning and Work (part 1)- Gram Consulting, June 15, 2009

   The goal of learning in the workplace is performance–individual and organizational.  If we’ve learned nothing else in recent years, we’ve learned that improving performance through learning is more effective the more it is integrated with real work.  Build a performance support system. Understand the job.

6. The Future of the Training Department- Learning and Working on the Web, October 21, 2009

   NCR delivered the first sales training. You’re going to shift the focus to creativity, innovation, and helping people perform better, faster, cheaper. Tags: Informal Learning Performance Improvement The latter 20th Century was the golden era of the training department. People had vocations, not jobs. The U.S. ASTD is born.

7. Bridging the Performance Gap: Training is Just Part of the Solution- Performance Punctuated, July 1, 2010

   If, on the other hand, you are changing your sales model, a pretty important and difficult change, you need a change management strategy and plan. Therefore, there always has to be some reinforcement and feedback to truly affect performance. Tags: Performance Improvement Talent Management Organizational Learning by Reni Gorman.

8. Social Learning at Sun- Learning in a Sandbox, June 5, 2009

   As a result of that they couldn’t provide the training hours needed to meet the training needs of the sales people. Secondly, sales people needed to have access to the right tools and information needed to close their deals in the field. informal learning??).

9. A Pharmaceutical Leveraging Web 2.0 --In a Big Way- Kapp Notes, November 17, 2008

   site containing information on mechanisms of action, employee and project profiles and Pfizer products and other information useful to researchers and sales personnel within Pfizer. If you think Web 2.0 can't be used in your company because of legal or regulatory concerns, think about this. Pfizerpedia is resource-sharing Web 2.0 tags: web_2.0

10. Sun Microsystems Uses Open Source and Enterprise Social Software to Build YouTube-Like Portal for Social Learning- Learning on the Leading Edge, March 19, 2009

More about → Sales eLearning – 21 Great Resources

Rapid eLearning Search [Stupid Google Results]

This is stupid but struck me funny.  I did a search for rapid eLearning and the following is a screen shot showing the results with arrows added for one that I really don’t understand.

There have to be all sorts of great jokes about what this implies about Rapid eLearning, Articulate, etc.  By the way, I had logged out at the time.  It’s NOT because of personalized search.

More about → Rapid eLearning Search [Stupid Google Results]

19 Tips for Effective Online Conferences

I was recently asked me about my recommendations for conducting effective online conferences (virtual conference).  They had attended the LearnTrends free online conference.  This conference has been run for a few years now and is tentatively scheduled for November 15-17 this year.

This is my attempt to collect some of what we’ve learned over the years doing those conferences.  There’s some good information back in Online Conferences and In-Person Conferences, Be an Insanely Great Professional Conference Attendee, Online Conference Formats, and Conference Preparation.

1. Assign people into roles.  For us we have someone who is an overall producer (actually there are three of us, but we split it well), session moderator, recording engineer, speaker prep person. 
2. Technical Issues.  Prepare for technical issues and decide what you will do about them.  We’ve used Elluminate as our webinar system.  It has issues with firewalls.  So, we ask everyone to test until they find a location where they can connect.  This means that some people need to attend from home. 
3. Communicate all of the details in great detail.  No matter how much you feel its obvious what to do, there will be confusion.  This is especially true with information like testing the virtual meeting software, getting into sessions, protocols during sessions, time zones, etc.
4. Closely related to the above, make sure you have a well known way/place to submit questions and some assigned to answer these.
5. Make it obvious where to access to resources from speakers.  For us, we create a discussion topic associate with the session and attach files or content into that.
6. Session timing and design is difficult online.  Most presenters are not used to designing for online, so it’s important to help them plan their sessions to be effective.  You definitely need interaction and time for Q&A.  Some of the best sessions have very effective means of capturing the wisdom of the attendees.  Even the length of sessions is challenging.  Long sessions can try to get too much content in and often lose steam before Q&A.  Short sessions often means really superficial.  This is where a good prep person along with a good moderator can really help.
7. Open discussions have worked, but they are risky.  I often find they meander too much for me.  Of course, I generally am more comfortable with directed Learning Goals.  Certainly, if you are going to try it have a great moderator and interesting audience.
8. Getting participation outside of the sessions is challenging.  People don’t read things, or enter discussions ahead of the sessions.  No matter how hard we’ve tried, it’s VERY difficult to get an audience to do anything ahead of when the sessions start.  And it’s even difficult to get them to engage through discussion forums outside of the sessions.  It’s best to design things where the discussion forums are an added bonus and have some set questions to spark discussion.  But don’t count on that much participation.
9. Make sure you have breaks between blocks of sessions.  The conference staff needs it, but so do the attendees, even if you think they will pick and choose sessions.
10. For us, defining the times for sessions is difficult.  There’s no way to choose time slots where it’s not an ungodly hour in some part of the world.  Not sure we’ve ever figured out an answer.
11. Text Chat works great during sessions as a Back Channel.  Encourage it and make sure the moderator and/or presenter weave it into the session – see Presentation Backchannel Multitasking.  Also, make sure that the presenter is not distracted by chat.  I always encourage speakers to ignore it and I’ll (as the moderator) let them know when there’s something I’d like them to discuss.  And I’m not a big fan of using multiple back channel tools that effectively split the discussion.  Just keep it in the webinar tool.
12. Inject fun into the event. Nancy White jumped in to lead a round of Pecha Kucha using slides she had never seen before. The spontaneity and spirit of fun raised energy levels. Need fun stuff sprinkled throughout.  Moderators can help this greatly.

13. Speakers on a common topic should get to know one another and swap ideas on their approach in advance of the session.
14. Pictures of speakers make the event more real. Have pictures on session descriptions and on the first slide of each speaker.
15. Thanks to Scott Skibell, we recorded the sessions directly off the web, i.e., outside of Elluminate and he produced great recordings.  This helps a lot.
16. We’ve tried a few different ice breakers.  Interaction to learn the webinar tool is always a good idea.  Sometimes a kind of introduction to break the ice with audio can be good.  Especially if you want audio input.  Many people freely text chat, but don’t like to speak online.
17. Be careful with screen sharing and video.  They require so much bandwidth that they often make the presentation not work for lots of attendees.  Much better to use screen shots.
18. Some people take online commitments lightly. Several presenters never showed up. Some volunteer moderators disappeared when we tried to pin down times for them to cover.
19. Online conferences have the advantage of being able to have introductory and truly advanced topics that will appeal to a narrow audience.

More about → 19 Tips for Effective Online Conferences

Evaluating Knowledge Workers

I’ve discussed before about how it’s Hard to Evaluate the Performance of Knowledge Workers and now some additional backup from Dilbert:

Which goes along with:

But let’s be careful here because:

More about → Evaluating Knowledge Workers