Academic Analytics At LSU

What is Academic Analytics?

Academic Analytics (AA) is a private firm providing data on faculty scholarly activity to PhD‐ granting universities in the US and UK. Founded in 2005 by then‐graduate school dean Lawrence Martin of Stony Brook University, AA has at least 35 AAU and 90 total subscribers plus over 70 employees working from locations throughout the US and UK. Website: http://www.academicanalytics.com/.

The AA database includes information on over 270,000 faculty members associated with more than 9,000 Ph.D. programs, 10,000 departments, and 6,000 additional academic units at close to 400 universities in the United States and abroad. To enable comparisons among performance data across universities, AA classifies faculty members into a taxonomy of approximately 170 disciplines, which are based on the National Center for Educational Statistics’ (NCES) Classification of Instructional Programs (CIP) coding scheme.

The database includes data on the following areas of scholarly accomplishment for faculty involved in Ph.D. education in the United States: journal articles; journal article and conference proceeding citations; book publications; conference proceedings; federal grants; and professional honors and awards. The data present research activity and citation primarily in relatively recent years; older publications and citations are not included. These records are collected by AA itself from publishers, granting agencies, and awarding bodies.  AA is engaged with higher education in an ongoing effort to expand the content of the database to provide a more comprehensive view of research activity across all disciplines.

It’s important to note what is counted in the Academic Analytics database and what is not counted:

What is counted in Academic Analytics:

  • Books (over 130,000 authored and edited books) (2006-2015)
  • National Honorific awards (6,039 honors/905 governing societies) (varies by award and ranges from 10 to 50 years)
  • Grants (Over 163,000 federal research grants/$33 billion/ 13 agencies – new and continuing) (2012-2015)
  • Entities and Taxonomy (9,700+ Ph.D. programs/11,000+ departments at 405 institutions)
  • Conference Proceedings and Citations (7,052 individual conferences) (2012-2013)
  • Journal Articles and Citations (6 million in 24,000 journals/ proceedings) (2012-2015)
  • 41,232 journals and 18,032 conferences

What is not counted in Academic Analytics:

  • Citations to books
  • Co-PI status on grants (federal)
  • Patents and patent disclosures
  • Industry and state sponsored research
  • Instructional teaching loads
  • Art displays or performances

Academic Analytics at LSU

  • Although AA provides data on individual faculty members, the majority of LSU users will only be able to review collective unit performance, not individual faculty records. The Office of Research and Economic Development can view individual scholarly records.
  • Access is granted to individuals in academic leadership positions on our campus. 
  • LSU holds a master license with Academic Analytics. Bernie Braun in the Office of Budget & Planning is the LSU data liaison to Academic Analytics.
  • A steering committee comprised of faculty, representatives from ORED, the Office of Academic Affairs and Budget and Planning meet annually to evaluate and make recommendations to the provost on the use and continued practicality of the product.

Guiding Principles

The Office of Academic Affairs, ORED, the Colleges, and all other administrative units with access to AA data will use it in conjunction with other sources of information available to them and with the full appreciation that AA’s data is one tool to use and should be used in conjunction with other data sources when making decisions.  

Keeping this in mind, we have provided the following guiding principles to direct actions and manage expectations.

  1. The primary focus of using Academic Analytics data is for continuous improvement. Where appropriate, however, the data may be used in conjunction with other sources of data for program assessment.
  2. Academic Analytics data are only one source of information. Programs are strongly encouraged to seek information from other sources.
  3. Access to Academic Analytics data requires individual training and agreement to follow Academic Analytics identified restrictions for use.
  4. In terms of access to the Academic Analytics data, the answer to the following question plays a significant role in determining this access: “does the individual have a managerial need to know?” In other words, do they maintain a level of authority over the unit that would enable them to initiate improvements within the program?
  5. The Office of Budget and Planning, as well as Academic Analytics, is committed to working with academic chairs to provide accurate data.
  6. When possible, deans and other subscribers will be given an opportunity review and respond to their unit’s data early in the process.
  7. Academic Analytics releases the most recent data in December of each year, based on the faculty of record during the previous fall term. (For example, Academic Analytics will release scholarly output data in December of 2017 for the faculty of record in fall of 2016.) The data released in December of each year will become the official data set used throughout the following calendar year (e.g., data released in December of 2017 will be the official data set used throughout calendar year 2018).

LSU Uses of Academic Analytics

  • Departmental and programmatic evaluation and expansion
  • Departmental peer comparisons and benchmarking
  • The evaluation of new program proposals
  • The analysis of potential cross disciplinary faculty hires
  • The identification of new research funding sources

Faculty Lists

LSU’s initial pool of faculty (Fall 2013) was generated from the Oct. 31 census date file. Included were faculty at the rank of Assistant Professor and higher from the LSU and Pennington Campuses, as well as faculty from the LSU Ag Center who were paid primarily by Ag Center departments that have equivalent LSU academic departments.

The primary selection criteria placed faculty in the academic departments paying the largest portion of their salaries. However, if a faculty member was cross charged to multiple academic departments, he/she was placed in the multiple departments for college review and approval. This methodology closely resembled data that has been continuously maintained by Academic Analytics prior to LSU’s participation. Faculty who were paid primarily by administrative units (e.g. senior administrators, deans, associate deans) were moved to academic units based on the tenure department or previous home departments.

Criteria for Future Faculty Lists and Validation Process

Academic Analytics provides guidance to determine which faculty to include. In consideration of these guidelines, we suggest the following criteria and process to identify faculty members at LSU.

  1. All faculty at the rank of Assistant, Associate and Full Professor, including faculty who are currently serving as senior administrators, deans, associate deans, and assistant deans etc. (Administrators with faculty rank will be designated by administrative title for the faculty submission.) Faculty are designated by tenure status (tenured, tenure track, not tenured).
  2. If a faculty member is joint-funded between/among more than one academic department, the faculty member will be included in each department as deemed appropriate by the colleges. The department paying the largest portion of the faculty salary will be designated as “primary”. Burden of proof is being “joint funded” in the financial records system.
  3. Tenured faculty who are paid by administrative units will be moved to the department in which their tenure resides. Those without a tenure department will be moved to an academic department as determined by the appropriate deans.

As new issues arise around the inclusion of faculty and the placement of faculty in departments, the Office of Academic Affairs will determine the appropriate action with input from the deans and chairs.

Faculty List Validation Process
The following steps will occur when the Office of Budget and Planning receives the annual request from Academic Analytics to update the faculty list:

  1. A list of faculty will be generated by the Office of Budget and Planning based upon the criteria mentioned above.
  2. The list of faculty for each PhD Program and academic department will be provided to the corresponding dean. The designated representatives in each unit have approximately three weeks to validate their list. If changes to the list are necessary, the deans will work directly with the Office of Budget and Planning to make the necessary changes. All changes must meet the guidelines put forth by Academic Analytics and the provost.
  3. After the three-week window, the list of faculty will be submitted to Academic Analytics.

Access Guidelines

Different levels of access have been determined:

Level I: Based on recommendations from the provost, specific members of the Provost’s staff, select staff from the Office of Research and Economic Development and the Office of Budget and Planning would have access to all campus program data.

Level II: All deans. Associate/assistant deans based on approval from their respective dean. Access would be granted to all academic programs in their school/college.

Level III: All department chairs based on approval from respective dean. Chairs would have access to their academic program, but not to other academic programs.

Requests for Access

Deans will submit a request in writing to Bernie Braun (bbraun@lsu.edu in the Office of Budget and Planning). Bernie will work with representatives at Academic Analytics to grant access and get log in credentials sent to the individual.

Required Training for Access

Academic Analytics requires that before access is provided to an individual, that individual must complete a WebEx training presentation. Academic Analytics will provide on campus training opportunities semiannually.

Current Users

Office of Research and Economic Development College of Science
Office of Academic Affairs    Budget and Planning
Art and Design             Coast and Environment
School of Mass Communication      College of Engineering
E.J. Ourso College of Business    College of Agriculture
Humanities and Social Sciences        School of Music & Dramatic Arts
Human Sciences and Education Office of Sponsored Programs


Concluding Thoughts

Despite the data imperfections in Academic Analytics, it does provide us with a tremendous amount of information that we did not have before. The key is to focus on what we can derive from such metrics and data and what we cannot. The careful use of technology and the availability of data certainly aid in the planning and assessment processes for our university, but omits other extraneous forces that help to shape our decisions—therefore it should be used with caution.


 

Non-disclosure conditions

Access to the LSU-Academic Analytics data will be granted only to individuals who complete and submit a non-disclosure agreement.  Conditions of our license with Academic Analytics stipulate that:
  • We will use AA data for internal analysis and decision making, to assist in the evaluation of departments and doctoral programs and to measure and compare faculty scholarship within LSU and amongst peers.
  • Content – AA both methods and results -- may only be distributed to and used by LSU employees, contractors and consultants (including members of the Board of Regents and Academic Review and Planning reviewers) who have a professional need or interest in using such Content for the purposes outlined above.
  • NO SCHOOLS other than LSU may be mentioned by name in discussing or presenting any results that may become public!!!
  • Public websites or external promotional communications of any kind referencing Academic Analytics data must be approved in advance by LSU authorized representative (Office of Budget and Planning) and by an authorized representative of Academic Analytics.
  • Every presentation or display of the Content or data derived from the Content must reference Academic Analytics as the source by stating “Fundamental data provided by Academic Analytics.”
  • Promptly following expiration of termination of the contract with Academic Analytics, all LSU units shall use reasonable efforts to permanently remove all copies of the content from its computer or other electronic systems and shall cease use of all materials that contain portions thereof, included all reports, presentations and charts.