Ways to Evaluate Educational Apps
Sunday, March 4, 2012 at 10:51AM 
I am conducting a series of workshops in Florida and was asked to share a rubric to help teachers evaluate educational apps as part of the workshop. In 2010 Harry Walker developed a rubric, and I used his rubric (with some modifications by Kathy Schrock) as the basis for mine. (Read Harry Walker's paper Evaluating the Effectiveness of Apps for Mobile Devices.)
I kept in mind that some apps are used to practice a discrete skill or present information just one time. Others are creative apps that a learner may use again and again, so it's a challenge to craft a rubric that can be used for a wide span of purposes. I tried to make my rubric work for the broadest range of apps, from drill and practice to creative endeavors, while stressing the purpose for using the app.
My rubric also emphasizes the ability to customize content or settings and how the app encourages the use of higher order thinking skills. Admittedly, there are good apps that are not customizable and focus on lower order thinking skills. Factor Samurai, for example, is a fantastic game for identifying prime and composite numbers. It would be nice if the app had flexibility to adjust difficultly, but it's still a good app if it is relevant to the learning purpose.
Here's what I chose to spotlight in my rubric:
Relevance
The app’s focus has a strong connection to the purpose for the app and appropriate for the student
Customization
App offers complete flexibility to alter content and settings to meet student needs
Feedback
Student is provided specific feedback
Thinking Skills
App encourages the use of higher order thinking skills including creating, evaluating, and analyzing
Engagement
Student is highly motivated to use the app
Sharing
Specific performance summary or student product is saved in app and can be exported to the teacher or for an audience
An app’s rubric score is very dependent on the intended purpose and student needs. The score you give an app will differ from how others score it. Again, apps that score low may still be good apps. But, it is handy to score apps if you are making purchasing decisions and/or have multiple apps to choose from.
Download the Education App Evaluation Rubric.
Perhaps more useful than a rubric is a checklist, so I developed one. I based my checklist on one created by Palm Beach County Schools and Edudemic.com. The checklist addresses both instructional and technical aspects of an app. For simplicity of purchasing, my list favors free apps and apps that do not have in-app purchases. Don't get me wrong, there are certainly fantastic paid apps.
The bottom line is what makes an effective app is one that does what you need it to do. And it's even better if it does it an inexpensive and engaging ways. There probably isn't an app that would receive all checks on my list, but in general, the more checks, the better the app is for education.
Here's my list:
- Use of app is relevant to the purpose and student needs
- Help or tutorial is available in the app
- Content is appropriate for the student
- Information is error-free, factual, and reliable
- Content can be exported, copied, or printed
- App’s settings and/or content can be customized
- Customized content can be transferred to other devices
- History is kept of student use of the app
- Design of app is functional and visually stimulating
- Student can exit app at any time without losing progress
- Works with accessibility options like VoiceOver and Speak Selection
- App is free of charge
- No in-app purchases are necessary for intended use of app
- App loads quickly and does not crash
- App contains no advertising
- App has been updated in the last 6 months
- App promotes creativity and imagination
- App provides opportunities to use higher order thinking skills
- App promotes collaboration and idea sharing
- App provides useful feedback
Download the Educational App Evaluation Checklist.
I welcome your comments as my thinking about what makes a good app, my rubric, and my checklist are all a work in progress.
Other educators have also put thought into evaluating educational apps. I'd like to point you to more rubrics and checklists.
Critical Evaluation of an iPad/iPod App is a yes/no checklist and has a place to write a summary of the app. It's by Kathy Schrock.
The Mobile App Review Checklist is from Palm Beach County Schools and Edudemic.com. It provides a yes/no checklist within Curriculum Compliance, Operational, and Pedagogy categories.
Mobile Application Selection Rubric is from eSkillsLearning.net and is a simple chart with criteria like aligned to Common Core Standards, Levels of Difficulty, and Various Modes of Play.
iEvaluate Apps for Special Needs is a detailed rubric specific for selecting apps for students with special needs. It's by Jeannette Van Houten.
iPad App Assessment Rubric for Librarians is from the Chicago Public Schools Department of Libraries. It's a Google Forms template you can use to collect app assessments.
Maybe more significant than evaluating the app itself is evaluating how the app supports instruction that infuses technology to create a powerful learning environment. The Arizona Technology Integration Matrix is a rubric for teachers to assess their level of technology integration across five elements of meaningful learning environments.
Arizona's matrix is based on the Florida Technology Integration Matrix. Like the Arizona version, Florida's features detailed explainations, videos, and lessons.
Please feel free to link to other rubrics and resources in the comments.
Reader Comments (18)
What a fantastic compilation! Implementation of iApps is moving at light speed and this will really give schools some tools that will help refocus on the purpose of the flashy, which is to enrich great teaching, not a substitute for it. Thanks!
Perfect timing! I have a group of educators getting together today to discuss iPads for students with special needs and I will share this post with them. Thank you!
Well done Tony. A concise list of rubrics from which educators will gain great value. At the present I am working with Special Education colleagues to develop "definitive" criteria for the selection of apps in the Special Education" environment.
Tony - these are great resources. The big complaint we're hearing from our schools is that there is no "try before you buy" option in the App Store, and even though15, 20, or 40 dollars seems a trivial amount for a school district with a budget in the millions, these purchases are frustrating in time lost deciding if the app is a fit or not. Any thoughts on a strategy for dealing with this problem?
Good post, Tony. As an education app developer (see Brainscape), I'm glad to see that educators are thinking about our products in the right way. But I think it's a bit unreasonable to expect an educational app to be free AND have no ads AND have no in-app purchase functionality. We developers have to spend hundreds of thousands of dollars (often using investors' money) in order to create these products. We need to earn money so that we can continue innovating and providing better products for teachers like you!
Also, I think the best way to think about apps (or games, in general) is to examine how efficient they are in terms of the time spent within the app. While some "frills" may be important to increase engagement, too many of them could decrease the amount of learning absorbed per every minute of usage. You may be interested in this post that I wrote about Evaluating the Efficiency of Educational Games. It applies to all education apps in general.
Feel free to share any thoughts with me at the email I've listed here in this comment, or find me on Twitter at @acohenNY
I truly agree that, "Maybe more significant than evaluating the app itself is evaluating how the app supports instruction that infuses technology to create a powerful learning environment." To that end, instead of rating apps, we run a session that focuses on the use of the apps (or potential uses) in the classroom or at home. We've posted that session for others as a self-paced, small group session:
Great way you have describe here ,It also will helpful to application developers to focus on developing educational applications.
Great post, Tony. I like your very thoughtful and comprehensive approach to selecting apps for a classroom. I personally strongly agree with the most items from your check list. And as an application developer (Geometry Pad) I have a goal to put a check mark across the majority items on the list. I would also like to join one of the people who commented and say that it is practically not possible to have a well made and completely free application. Application development involves spending thousands and tens of thousands of dollars even if you are an independent developer developing the app in your free time. It includes cost of coding, work of designers, Apple subscription fees (which is $99 per year by the way), web site hosting and domain registration, efforts to market and support the app. Also, the margin is very tight. Thanks to Apple, all iPad applications are way under priced. You pay just a couple of cups of coffee or less for an application that required months of development. I know that a lot of developers are struggling to break even or loosing money on app development.
I understand that you meant free price tag is a nice to have but not a strong requirement. I just wanted to share some developer's perspective on the subject.
Tony your post is a wonderful resource. Good criteria for evaluation but also a range of useful elements for designing educational apps.
Logical and practical! Thanks for sharing!
Please excuse my ignorance, but is the Rubric for the person who is considering purchasing an app or for the person who has purchased an app and wants to evaluate it. Usually an evaluation is done after one has experienced an event of some sort.
If the purpose of the Rubric is to allow me to make an informed decision on what app may work best for my needs, I would find a Rubric that had been filled out more useful than a blank one. Some of the categories like, “is it engaging, it’s relevance” is hard to determine unless you’ve used the app, which would entail the purchase of the app thus making the Rubric unnecessary.
The only way I see a Rubric saving anyone time and money is if a whole bunch of them were filled and made available on a website so that potential buyers could make informed decisions on which app to buy. Thank you to those people, who purchased an app that fell short of expectation, for filling out a Rubric so others could save time and money.
On the other hand a checklist of desirable features to for in an app would be a great thing to have.
Tony
A fantastic complilation. Thank you. I have been presenting at various State Technology Conferences and look forward to adding some of your information to my workshop.
In reply to the comment about "try before you buy". That is a very valid comment. As a publisher, we are given a number of promo codes that we can distribute for that purpose. But, the quantity is very limited. Also, teachers have asked us to "bundle" at a reduced price - that is offer a package. The problem is that none of those options fit into the "Apple model", so publishers do not have those options. Any suggestions?
Wonderful collection. I've been struggling to put together what you shared in a single post. Thanks.
Thank you very much!This is a great compilation exactly what I need now.As a Chinese language teacher,I am doing the reserch about how APPs help students learn to write Chinese Characters.
Hi,
Please take look at www.educationalappstore.com.
The Educational App Store is THE marketplace for all mobile educational apps. We bring you educational apps for all mobile platforms including iPhone, iPad, Android, Blackberry and Windows Phone.
With a huge range of apps available selected by the Educational App Store team, students, teachers and parents can browse apps by subject, age range or platform to find the perfect app for their needs
Nice job - as I am just now exploring the world of classroom apps, this will come in very handy! Thanks for your dedication.
Great resource, Tony. I especially like the evaluation for special needs apps. I spent quite a bit of time looking for just such a vehicle before I found it here.
I found this blog very interesting so I have bookmarked it. I hope you do not mind if I refer to your blog on mine thank you Richard