Harnessing .NET as a Framework for Assessment Data Management - Developing PARS - Program Assessment Reporting System

Harnessing .NET as a
Framework for Assessment Data
        Management
    Developing PARS – Program
   Assessment Reporting System

                  Jeffry Babb
       Department of Information Systems
       Virginia Commonwealth University

Outline
• The need for an information system for
  assessment
  – Building our own vs. COTS
  – Efficient means for RAD
• Defining the process
• Harnessing the .NET application
  framework - PARS
• Demonstration
• Conclusion and Discussion

Developing PARS

Why do we need an information
            system?
• Program assessment is an intricate and detailed
  set of activities and processes
• Many inputs and people must be managed over
  time
• Reporting, analysis and improvement activities
  must weigh and orchestrate multiple factors
• An assessment information system acts as a
  governing and managing agent in parallel with
  and in support of faculty efforts

What is an Information System?
• Information System?
  – People (students, faculty, staff, ABET, etc.)
  – Tasks (assessment metrics, reporting,
    improvement, analysis)
  – Applications (forms, reports, assessment
    data, measures, etc.)
  – Collect, process, transmit and disseminate
    assessment data towards a continuous
    process of improvement

Goal: manage the dual assessment
            processes

Detect and Assess   Respond and Improve

Developing an Emergent
        Information System
• People ↔ Processes ↔ Assessment Data
• Reconcile the loops
  – Program Assessment Effort (Detection Loop)
  – Program Analysis Effort (Reflection Loop)
  – Program Quality Improvement Effort (Action
    Loop)

An Information System to Manage
     all Assessment Activities

                   Act

          Detect         Reflect

Defining our Process
1.   Align program goals with the University and School
     mission statement
2.   Clarify program goals and program outcomes
3.   Clarify compatibility with ABET objectives and
     outcomes
4.   Match course outcomes to program outcomes
5.   Develop program assessment implementation and
     monitoring strategy
6.   Analyze assessment data and suggesting corrective
     actions
7.   Reflect on the program at large (what is assessment
     data telling us about ourselves?)
8.   Develop and adhere to an operational schedule

PARS – Program Assessment
       Reporting System
• A .NET-based information system for ABET
  program assessment activities
• Timeline
  – Planning and design – Summer 2006
     • Reflection on the findings of a faculty sub-committee on
       assessment from 2005/2006
  – Initial development – Fall 2006
     • Faculty-directed student work
     • Some setbacks (student and staff turnover)
  – Final development stages – Spring and Summer 2007
     • Faculty-directed student work

PARS – Key Elements
• FCARs
  – Rubrics
  – Metrics
• Surveys
• Testing
• Capstone course embedded
  measurements
• Reporting

FCAR – Central Instrument
• The Faculty Course Report is a central tool for gathering
  metrics on course outcomes.
   – The FCAR is well established as a useful tool
• FCAR maps course outcomes to program outcomes
   – How does each course contribute to the final results?
• Courses are where the “rubber meets the road”
   – The majority of program outcomes are effected and affected
     here
• The FCAR
   – Records Student performance
   – Records Faculty impressions
   – Records Student impressions

Assessing Artifacts – Student
     Group and Project Work
• Focus on group and project work
• Rubrics help to align program outcomes
  and course outcomes
• Reflective of program objectives
  – group and project work is reflective of
    professional activity
• Measurement scale – 3 points – Excellent,
  Average, Poor

Surveys
• Alumni survey
  – 1, 3 and 5 years
  – Alumni relationship building
• Industry Advisory survey
  – Employers
  – Interested community businesses
  – Track trends which affect program objectives and
    program outcomes
  – Are we developing a “product” that business find
    useful?

Harnessing the .NET Framework

Why build it ourselves?
• There’s plenty of COTS out there: MAKTEAM’s
  EvalTools, rubrics.com, Perception, etc.
• Will these fit US?
   – Yes
      • Makteam’s stuff is quite good
   – No
      • IS program certification is new
      • A “tailored suit” is a better long-run solution
      • We need a “long-run” solution as assessment is a long-term and
        ongoing process
• Okay, if we are rolling our own, how can we do this
  quickly?
   – .NET as RAD approach

Harnessing the .NET Framework
• Microsoft’s comprehensive application/system
  development framework
• Provides a full set of tools for RAD web
  development
  – ASP.NET and Visual Studio
• Provides data federation and management
  – SQL Server, SQL Server Reporting Services and
    Active Directory Services (LDAP)
• We are familiar with it…

Why .NET for RAD?
• We are familiar with it…
  – Imagine Cup US National Winning teams in 2005 and
    2006
  – We are moving towards C# and .NET for our
    application development courses
  – We actively participate in Microsoft’s student
    ambassador program
  – We actively utilize our MSDN academic alliance site
    license
  – Richmond metropolitan area IT firms seem to favor
    MS-based solutions

How does .NET provide for RAD?
• MSDNAA
   – Provides the necessary tools
       • Windows 2003 server, IIS, .NET runtime, ASP.NET, Visual studio,
         SQL Server, ADS
• Authentication
   – Single sign-on using ADS
• RAD Web Application Tools
   – Visual studio, ASP.NET, ASP.NET web services, IIS, FPSE
• Seamless library providing a full API
   – The .NET Framework Class Library is among the most
     comprehensive available (rivals Java’s API Class Library)
• Reporting
   – SQL Server Reporting Services greatly simplifies report
     generation (akin to Crystal Reports)

What is .NET?
• Microsoft’s comprehensive, web-centric,
  application development and delivery platform
• Interpreted and JIT compiled
• Targets many languages (VB.NET, C#, C++,
  Java, others)
• Folds applications in on one runtime
  environment and one API library
• Focuses on web applications and web services

.NET Applications

Typical .NET scenario

ASP.NET
• Application Server Pages .NET
  – Brings an event-based desktop application
    model to stateless and asynchronous web
    application development
  – Very easy to use
  – Application logic is not embedded within the
    presentation layer
  – Visual studio provides a drag-and-drop
    approach to page/application development

ASP.NET Overview

ASP.NET
• Let me show you how easy it is…

Single Sign-on Authentication
• We are able to simplify authentication and
  tie into other department resources
  – Active Directory Services
  – ASP.NET Membership providers
• Integration with student and faculty
  credentials
  – Expand into online testing and assessment
    collection

ASP.NET Membership Provider
• ACL at the page and page-element level
• Would allow various levels of access to
  the system
  – Administrator
  – Department Chair
  – Faculty
  – Support staff
  – ABET visitor
  – Other

ADS for integration with other IT
     and instructional assets
• Active Directory Services
  – Based on LDAP
  – Single Sign-on
  – Integrates well
• Departmental uses
  – Student labs
  – Instructional computers
  – Back office elements

SQL Server Reporting Services
• Provides a RAD approach to reporting
  – Many templates and wizards are available
• Federates data sources
  – Synchronizes MSSQL, XML, Office document and
    other sources
• Integrates well with ASP.NET
  – The .NET Framework Class Library provides routines
    for programmatic access
• Available through our MSDNAA license
  – The entire set of tools are readily available to us

SQL Server 2005 Reporting
        Services

SQL Server Reporting Services
        Architecture

PARS in action
• Still in development
• Here’s a sample…

Conclusion and Discussion
• THANK YOU!!!
• Discussion
  – Questions on how to get started with .NET?
  – Questions on the MSDNAA?
  – Discussion: Have YOU created an
    assessment information system?
    • Do you have any advice?
    • Do you have any questions?
    • Can we learn from each other?

Contact
• If you have any questions or suggestions
  at a later time, please contact me. ☺

 Jeffry Babb – jbabb@isy.vcu.edu
 Department of Information Systems
 Virginia Commonwealth University