Learning Systems Design and Development Competencies

This LSDD competency forms the reasoning and justification for creating learning systems. Needs assessment represents the beginning of the design process and serves as the foundation on which learning systems are built. A systematic investigation illuminates performance problems and learning needs. Careful data collection and accurate data analysis lead to better design. Needs assessment is the most essential aspect of a learning system since it supplies the reason for further design and development.

While needs assessment shows how a potential system could be used ideally, evaluation reflects the reality of it. Evaluation establishes the new system's significance and how well it fits into the organization or learning community. It demonstrates how the learning objectives identified in the needs assessment were met or unmet.

If the time and budget allow iteration, a formative evaluation is useful so the system can be adapted. After implementation and investment into a new system, designers need to reflect back to see how well it works. A summative evaluation offers a perspective on a system's worth and how meaningful and useful it is for the learners.

Demonstration of Competency: Independent Bookstore Needs Assessment

In 2006, I individually conducted a needs assessment on an independent bookstore retailer and examined the performance problem of below-quota sales for one genre of books, bestsellers.

The final report (PDF) includes the identification of the problem/opportunity statement, description of the organization and its context, results of data collection, identification of staff skills and recommendations.

Primary data were collected from interviews, surveys and observations. Two one-on-one interviews were conducted using a detailed 32-question protocol. The interview data were transcribed and sorted within the Performance Pyramid categories to internal and external factors.

Nine participants were invited to take a seven-question online survey (survey screenshot); six responses were returned. Survey responses were received via email. The nominal scale responses of "office or sales" were compiled. Text answers for the "feeling" section were compiled and assigned a general value of positive, negative or neutral. In the "ranking two tasks" section, percentages of responses were applied to the first and second places.

In the "improvement factors" section of the survey, responses were ranked with the most highly-chosen factor at the top. In the "knowledge of bestsellers" section, importance and knowledge ratings were transferred into a Task Matrix. Long text comments at the end of the survey were grouped into elements of the Performance Pyramid and examined for relationships and similarities.

Two one-hour observations were conducted using a two-page observation guide. During the observation periods, notes were marked for each section with details. These notes were reviewed and analyzed qualifiedly using the elements of the Performance Pyramid.

Secondary data were gathered from financial reports on the bookstore and book retailers as an industry.

Detailed summary results from the interviews, surveys and observations are available in the final report. Overall, the data showed that the primary factor contributing to the performance problem within the control of the bookstore was the tools, environment and processes element of the Performance Pyramid. At times, sales staff members could not see the customers.

If I were to do this project again, I would ask a broader question about the performance problem. Although it was true that the numbers showed a performance problem in bestsellers, I learned through the process of the needs assessment that this bookstore should not be focused on bestsellers. The market niche where the bookstore fits is better suited to more specialty items, such as fringe pop material and cookbooks.

Multiple aspects of my recommendations were adopted; however, results were not outstanding due to the fact that the identity of the bookstore is not in line with being a primary source of bestsellers.

If I had had a better understanding of the role of independent small bookstores vs. national chain and internet booksellers, I would have asked who the audience was before looking at such a specific performance issue that was not viably solved in the end.

I would have also examined the factors of the economy more deeply since retail sales are so closely linked to consumer confidence.

Overall, if I were to do this project again, I would ask broader questions about performance for a longer time before narrowing it down.

Demonstration of Competency: ADD-IT Project Evaluation Plan

As part of Formative and Summative Evaluation in 2007, I worked all semester as part of a four-person team to create an evaluation plan for the Web site of a grant project. Our final project was the Achievement through Data-driven Decision-making and Instructional Technology (ADD-IT) Evaluation Plan (Word).

With both formative and summative components, our plan included an examination of the usability and the effectiveness of the site in meeting its grant objectives.

Our team developed a solid, detailed plan and I don't have any changes in mind for it if I were to do it again. Based on teammate Cat Boedenauer's idea, one change I would make to the creative process would be to use a different collaborative tool where the document could be kept online and easily edited by all of us. We maintained a wiki throughout the semester which was helpful but at times unwieldy. I have since been exposed to tools during my CSCL and CSCW classes that I think would be better suited to this process.