Jordan Barrish suggests when selecting your first Learning Management System (LMS) to: "Think Through Your Courses" and "Focus on Usability" ("Two Key Steps to Selecting and Loving Your First LMS", eLearning Industry, 26 August 2013). I suggest that the first of these is most important the second applies to any computer based application and should be a matter of routine.
Many make the mistake of selecting a LMS and then trying to use that to develop e-learning courses. The LMS is just a framework to put around the e-learning courses to help administer them: delivering the materials to the student and recording progress. The LMS is not intended for creating the actual course content.
Most LMS provide some content creation tools, but these are crude at best. Courses should be designed outside the LMS using specialist tools and then installed in the LMS for delivery. If you use the LMS to create the courses, then be prepared to use very simple formats and have to do much manual work to overcome the limitations of the tools.
As LMS are not intended for creating courses, tinkering with an LMS is not a good way to learn teaching or course design. Staff designing and delivering courses need training first in how to teach and then in how to teach on-line. Just giving a university lecturer who has never had any formal education in teaching an LMS is a recipe for frustration, for other the staff and students. Sending staff on a course on how to use the LMS will not help, as the LMS have built in assumptions about pedagogy and course design. Staff need to learn to teach, learn to design e-leanring and then using a LMS is simple.
Usability can be evaluated first by use of the World Wide Web Consortium Web Content Accessibility Guidelines (W3C WCAG) with someng like A Checker, Mobile device compatibility (W3C mobileOK Checker) and Markup Validation Service. These will indicate if the tool implements web standards and will be usable on a wide range of devices and network speeds.