2023-24 College Accountability Phase 2 (Scorecard) - Project Status, Decisions, Risks and Meeting Notes
Jump to Meeting Agenda and Notes
Project Status
Phase 2 - Design and creation of College Scorecards. COMPLETED
Project Timeline
Last updated and approved by Valeria on 09.07.2023.
Mock-up and approval of College Scorecards (Aug 18 - Sept 22) Completed
Website modifications and approval and CMS modificaitons (Nov 9 - Nov 14) Completed
Development and QA of Scorecards (Sep 20 - Nov 9) Completed
Go Live for College Scorecards (Nov 14 - 15) Completed and went live on Nov. 17 at 1pm
Project Watch List
Citation Impact for Phase 1 dashboard and Phase 2 scorecard. No way to capture this at the college level, but Jay is working with Strategic Metrics to get this information.
Technical documentation (Action Item from Retrospective) - Not specific for phase 2, keep it as a separate issue.
These three metrics will not be available until March, so no need to discuss until then.
Phase 1 and 2 - Non-Medical Science & Engineering Research Expenditures ($M) - Not available until after 3/1/24
CA Phase 1 and 2 - Science & Engineering Research Expenditures ($M) - Not available until after 3/1/24
CA Phase 1 and 2 - Effective F&A Rate (%): By College (Effective F&A Rate) - Not available until after 3/1/24
Meeting in March to start discussion on Phase 3 scope and timeline.Later phase - migration to Microsoft Fabric for Point in Time Dashboards
Meeting Agenda and Notes (most recent on top)
1/26/2024 Status Meeting Agenda/Notes
Citation Impact - Valeria is talking to USF World (Kiki) to determine next steps.
Waiting on technical documentation and March 1 to discuss other issues.
Open Discussion
Sangita has moved the disclaimer “2024 Data is incomplete” to the top of the page. Valeria asked that it be in red.
Sangita showed SSRA calendar (refreshes, etc.).
1/18/2024 Status Meeting Agenda/Notes
Review Watch List - Next week, it was just reviewed on 1/11/2024 meeting.
Open Discussion
Sangita showed scorecards to confirm Total Fundraising has been moved. Valeria will send the Deans an email to let them know that Total Fundraising is now available.
Sangita prefers not to do an asterisk and instead have the note on the top as the "maintenance" (back end implementation) is easier; there are many reports with many tabs.
Faculty Awards - Sangita was asked to break out TARU, NRC, etc. are broken out into individual items, not TARU and then TARU/NRC… in the drop down for Faculty Awards.
Retention needs a detailed discussion, Shruti, Valeria, Sangita, Glendalis, Steve, Jay, Leslie and Kim on this meeting. Shruti will have Tammy schedule.
Sangita to meet with Iyshia on Faculty Awards metric.
1/11/2024 Status Meeting Agenda/Notes
Standardize Metrics Names between projects, including College Accountability. Listed as a separate project, but will start soon. Sarah, Shruti and Valeria will discuss.
Citation Impact for Phase 1 dashboard and Phase 2 scorecard. No way to capture this at the college level.
Total Fundraising - Meeting was completed with Susan Vandermast & Noreen Seagrest. They have agreed to provide the data but have not heard back. Valeria will follow up. They have fundraising data now.
Documentation
Refresh schedule complete.
Received information on Invention Disclosures?
Technical documentation (Action Item from Retrospective) - Not for phase 2, but add it as an issue, but not specific for phase 2, keep it as a separate issue.
These three metrics will not be available until March, so no need to discuss until then.
Phase 1 and 2 - Non-Medical Science & Engineering Research Expenditures ($M) - Not available until after 3/1/24
CA Phase 1 and 2 - Science & Engineering Research Expenditures ($M) - Not available until after 3/1/24
CA Phase 1 and 2 - Effective F&A Rate (%): By College (Effective F&A Rate) - Not available until after 3/1/24
Further meetings to track these items above or hold until March? Keep these as is and at the same time. If we do not need we will cancel.
Meeting in March to start discussion on Phase 3 scope and timeline.
12/14/2023 Status Meeting Agenda/Notes
Meeting repurposed for Retrospective. See separate link for retrospective notes in project dashboard.
12/05/2023 Status Meeting Agenda/Notes
Retrospective for Phase 2? Yes
Address Watch List:
Standardize metric names - ask Valeria across project/dashboards - Yes, we need to do this.
Documentation for College Accountability - Working on refresh schedule, should be done for holidays. Need to get information on Invention Disclosures.
CA Phase 1 - Fundraising - yes (Total Fundraising) - DONE
CA Phase 1 - Public University National Ranking - Do not have at the college level - No work needed
CA Phase 1 - Citation Impact - (See below)
CA Phase 1 - Non-Medical Science & Engineering Research Expenditures ($M) - Not available until after 3/1/24
CA Phase 1 - Science & Engineering Research Expenditures ($M) - Not available until after 3/1/24
CA Phase 1 - Overall F&A Rate (%): By College (Effective F&A Rate) - Name change is done, however, data is not available until after 3/1/24
Follow up from last meeting
Citation Impact (upcoming) - Timeline? Waiting on a different group to get information
Annual Fundraising (upcoming) - Was meeting set up and/or held with Valeria, Susan Vandermast & Noreen Seagrest? Meeting was done. Timeline? Valeria/Jay said that they have agreed to provide the data but have not heard back. Valeria will follow up.
Hide Effective F&A Rates page - Deena in Research will get back to us with updates and let us know when we can “unhide” this page. Not before 11/15. Timeline? (See above)
Deena to give information on discrepancies to be able to explain and Sangita will add this to online documentation. Not before 11/15. Timeline? DONE
Faculty awards will be distributed when it is available; need to establish a goal for total awards. If TARU is around for next year, then we can set a goal for this subset of awards. Keep on watch list or establish timeline? DONE
Faculty Awards is being updated manually - Do we plan on automating this or is this good for now? Keep this manual, no work needed.
11/06/2023 Status Meeting Agenda/Notes
Meeting is to allow Q&A for Sangita and her team.
11/02/2023 Status Meeting Agenda/Notes
Project Schedule
Mock-up and approval of College Scorecards (Aug 18 - Sept 22) Complete
Website modifications and approval (Oct 2 - Oct 20) Hennadii - On Track to start
Development and QA of Scorecards (Sep 20 - Nov 13) Sangita - On Track
Go Live for College Scorecards (Nov 14 - 15) - Team - On Track
Discussions
Watch List
Valeria will resend the email to Susan Vandermast & Noreen Seagrest for Annual Fundraising metric and goal. Meeting is going to be set up.
Grant dollars and Research Expenditures, Grant proposals submitted.
Hide Effective R&A Rates page - Deena in Research will get back to us with updates and let us know when we can “unhide” this page. Not before 11/15
Deena to give information on discrepancies to be able to explain and Sangita will add this to online documentation. Not before 11/15.
Documentation online (including refresh dates). Refresh calendar updated in Confluence, updates needed?
Have all changes for the scorecard format been completed? Done.
On Track for Nov. 7th. Not including Hennadii’s work which is a quick change. Sarah’s CMS change is also fast. Still on Track? No, changed final due date to Nov. 13th. 8 out of 12 metrics are done for scorecards. All colleges are done for each metric.
The CA reports completed during phase 1 will be refreshed. 80% are done. Research is noted above with data still changing. Complete by Monday, 11/6. Dependent of Fall preliminary bench.
Work this past week
Scorecards and the total research expenditures and refresh for the metrics for phase 1.
Work next week
Complete scorecards by Nov. 13th. Go live the week of Nov. 13-17th.
Open Discussions/Questions
CMS web site modifications
Sarah to create a Box folder to share screen shots and send out link.
Let everyone know that Advancement awards will be distributed when it is available. Valeria will send this information. Need to establish a goal for total awards. If TARU is around for next year, then we can set a goal for this subset of awards.
Show Total Awards Goal and for OneUSF (sum of all the colleges).
Tenure track FTE is during a fiscal year. Numerator and denominator will both change. No goal as it is a completely new metric.
Shruti shared the document she has created for the refresh schedule. Currently in a spreadsheet, but she can transfer to another format.
Two sets of scorecards, one for the colleges and the other for One USF.
The faculty awards metrics is being updated manually right now.
Give screenshots to Deans on Monday 11/13
Go live week of 11/13-17
10/26/2023 Status Meeting Agenda/Notes
Project Schedule
Mock-up and approval of College Scorecards (Aug 18 - Sept 22) Complete
Website modifications and approval (Oct 2 - Oct 20) Hennadii - On Track to start
Development and QA of Scorecards (Sep 20 - Nov 7) Sangita - On Track
Go Live for College Scorecards (Nov 14 - 15) - Team - On Track
Discussions
Watch List: Valeria will resend the email to Susan Vandermast & Noreen Seagrest for Annual Fundraising metric and goal. Meeting is going to be set up.
Watch List: Grant dollars and Research Expenditures, Grant proposals submitted.
Goals (Sangita)
Some of the goals are blank, Valeria will follow up and has sent an email/note to EDU and MAR. SCI regarding these empty goals. DONE
Hide Effective R&A Rates page - Deena in Research will get back to us with updates and let us know when we can “unhide” this page. UPDATES? Not before 11/15
Deena to give information on discrepancies to be able to explain and Sangita will add this to online documentation. Status - Not before 11/15.
Sangita is tracking what data is changed when and will put into the CA Team meeting chat and tag Valeria. Have all changes for the format discussed last week been implemented and approved? Working on it now.
Valeria said we need to give the Colleges a few days to preview the Scorecards before they go live. WE NEED TO HAVE ALL DONE BY NOV. 9th, so that Valeria can send out to Colleges to preview.
On Oct. 13, Sangita said that the Scorecards should be done in 3 weeks, by Nov. 3rd. On Track for Nov. 7th. Not including Hennadii’s work, quick fix.
The CA reports completed during phase 1 will be refreshed. 80% are done. Research is noted above with data still changing. Status?
Refresh Documenation (Timeline?)
Sangita explained that she is putting refresh dates with the data and will demo this in the future.
Shruti showed everyone the Confluence page that has the refresh information. Kevin is updated and should be done when?
Work this past week
Scorecards and the total research expenditures and refresh for the metrics for phase 1.
Work next week
Scorecards and the total research expenditures and refresh for the metrics for phase 1.
Open Discussions/Questions
10/19/2023 Status Meeting Agenda/Notes - Meeting cancelled.
10/13/2023 Status Meeting Agenda/Notes
Project Schedule
Mock-up and approval of College Scorecards (Aug 18 - Sept 22) Complete
Website modifications and approval (Oct 2 - Oct 20) Hennadii - On Track to start
Development and QA of Scorecards (Sep 20 - Nov 9) Sangita - On Track
Go Live for College Scorecards (Nov 14 - 15) - Team - On Track
Discussions
Hennadii - not a large amount of work and can be done towards the end of the project.
Valeria will resend the email to Susan Vandermast & Noreen Seagrest for Annual Fundraising metric and goal. Meeting is going to be set up. Valeria asked to keep this on the watch list.
Goals (Sangita)
Some goals are empty - What do we do with these? Sangita will be filling these in according to previous year, etc. (same technique that has been done in the past.
Would like to verify goals with colleges - FTIC retention at 93%, only BCS has FTIC Ret at 93%. This will be done with the colleges according to Valeria. Some of the goals are blank, Valeria will follow up and has sent an email/note to EDU and MAR. SCI regarding these empty goals.
Hide Effective R&A Rates page - Sangita was directed to do this by Deena/Research.
Sarah to park this issue and revisit. Grant dollars and Research Expenditures, Grant proposals submitted.
Deena to give information on discrepancies to be able to explain and Sangita will add this to online documentation.
Sangita is tracking what data is changed when and will put into the CA Team meeting chat and tag Valeria.
Valeria said we need to give the Colleges a few days to preview the Scorecards before they go live.
Research metrics
Access issue with PowerBI dataset, no one can see data. Sangita working with IT and Deena to get this fixed. Resolved.
Research metrics give a “gateway error” every time it is refreshed. IT is working to resolve. Resolved.
The CA reports completed during phase 1 will be refreshed. 80% are done. Research is noted above with data still changing.
Refresh Documenation (wait until Next Week)
Sangita explained that she is putting refresh dates with the data and will demo this in the future.
Shruti showed everyone the Confluence page that has the refresh information. Kevin is updated and should be done when?
Work this past week
Scorecards with Research and Endowment
Work next week
Scorecards should be done in 3 weeks.
Open Discussions/Questions
Valeria asked to bring up the mock to quickly review.
10/05/2023 Status Meeting Agenda/Notes - Meeting Cancelled No Updates
09/28/2023 Status Meeting Agenda/Notes
Project Schedule
Mock-up and approval of College Scorecards (Aug 18 - Sept 22) Complete
Website modifications and approval (Oct 2 - Oct 20) Hennadii will take on responsiblity (Chirag can assist if needed)
Development and QA of Scorecards (Sep 20 - Nov 9) On track
Go Live for College Scorecards (Nov 14 - 15)
Discussion
Valeria will resend the email to Susan Vandermast & Noreen Seagrest for Annual Fundraising metric and goal. Meeting is going to be set up. Valeria asked to keep this on the watch list.
Modifications for scorecard mock-ups. None needed all scorecard mockups have been approved.
Grant dollars awarded will be connected to the IT central powerbi dataset for pulling in the numbers. Any follow up needed?
Remaining CA reports which are manually updated will be refreshed this week. The CA reports completed during phase 1 will be refreshed.
Work this past week
Working on College Scorecards
Work next week
Continue working on College Scorecards
Open Discussions/Questions
Build out a timeline document similar to the PPA version and add it to the CA website. Timelines need to include refresh cycles etc. Potentially look at push notifications to let colleges know. Sangita explained that she is putting refresh information with the data and Shruti showed everyone the Confluence page that has the refresh information. Sangita will show the team an example of the documentation in the future.
09/20/2023 Status Meeting Agenda/Notes
Project Schedule
Mock-up and approval of College Scorecards (Aug 18 - Sept 22)
Website modifications and approval (Oct 2 - Oct 20)
Development and QA of Scorecards (Sep 20 - Nov 9)
Go Live for College Scorecards (Nov 14 - 15)
Review work done on updated mock-up of Scorecards
Decisions we made last meeting. No need to review progress, just as an FYI for the team.
Separate tab for the “OneUSF” grouping of metrics.
The plan is to add a single link called “Scorecards” on the main College Accountability landing page.
Work this past week
Link the Faculty Awards page (Taru report) on the research website as documentation from ODS’s site has been done.
Modifications for scorecard mock-ups
Research metric of “Overall F&A rate” has been modified to “Effective F&A Rate” based on Deena’s suggestion.
Work next week
Grant dollars awarded will be connected to the IT central powerbi dataset for pulling in the numbers.
Remaining CA reports which are manually updated will be refreshed this week.
Open Discussions/Questions
Build out a timeline document similar to the PPA version and add it to the CA website. Scorecards will replace PPA. Need to figure out where the scorecards will appear on the CA website. Timelines need to include refresh cycles etc. Potentially look at push notifications to let colleges know.
Valeria settled on the all consistent green color for the scorecard for now.
Goals for TARU & Others metric was not asked for from Research and will need to be accommodated in the scorecard.
Valeria will resend the email to Susan Vandermast & Noreen Seagrest for Annual Fundraising metric and goal.
In October for the first release there will be no “Previous Year” button.
09/14/2023 Status Meeting Agenda/Notes
Project Schedule
Mock-up and approval of College Scorecards (Aug 18 - Sept 22) - On track
Website modifications and approval (Oct 2 - Oct 20)
Development and QA of Scorecards (Sep 20 - Nov 9)
Go Live for College Scorecards (Nov 14 - 15)
Review work done on updated mock-up of Scorecards
Faculty awards tab change from percentages to numbers - Progress?
Decisions we made last meeting. No need to review progress, just as an FYI for the team.
Note that we will add OneUSF scorecard
Note that we will have space for 2 buttons, one now to go back to the 5-year trend and another button for the prior year that will eventually be the point in time.
Note that we need to look at the mobile view template in PowerBI
Work this past week
Received feedback from Valeria, Jay and SSRA built mockup showing scorecard with the feedback in place.
Work next week
Modifications for scorecard mock-ups
Open Discussions/Questions
Valeria pointed out a need for ADA compliance and color schemes. Sangita is reviewing it and explore the options. Not a priority but would recommend doing a due diligence on ODS’s side. Shruti volunteered to work with Stephanie to find out more about it. Review by next meeting.
On mobile, the view will show one card at a time as per Sangita.
Discussion on whether to go for 6x2 format (portrait) or 2x6 (landscape) format will be based on how it appears when printed on the page. Preference is portrait mode (6x2). Will have done by next meeting.
Groupings by color was discussed and Valeria was not sure if we needed it. Perhaps having groups of colors by domain made more sense. SSRA will built mockups showing the options next week
Faculty awards is counts and no percentages. Current year was done as pct by mistake. This has been fixed and accounted for in next year’s cycle. For this year, Valeria has already explained this to the colleges.
The plan is to add a single link called “Scorecards” on the main College Accountability landing page.
Separate tab for the “OneUSF” grouping of metrics.
Academic Affairs and Health breakouts may be needed. For certain metrics this grouping may be a challenge and would need to be investigated (e.g. Others). Valeria explained how the Provost would like to see this distribution especially on items like Research related metrics. SSRA will investigate and come back next week with recommendations.
Research dataset will be online and pulled in starting next week. Last update (done manually was back in July) but now will be automatically pulled in.
Valeria suggested to link the Faculty Awards page (Taru report) on the research website as documentation from ODS’s site.
09/07/2023 Status Meeting Agenda/Notes (Shruti provided a new schedule below)
Agree on Schedule (Original Proposal)
Design and development of College Scorecards (Aug 18 - Sept 29)
Website modifications (Oct 2 - 11)
QA and approval of Scorecards (Oct 9 - 13)
Go Live for College Scorecards (Oct 20)
Proposed revised schedule from Shruti. This schedule was approved by Valeria
Mock-up and approval of College Scorecards (Aug 18 - Sept 22)
Website modifications and approval (Oct 2 - Oct 20)
Development and QA of Scorecards (Sep 20 - Nov 9)
Go Live for College Scorecards (Nov 14 - 15)
Review work done on updated mock-up of Scorecards - Must be able to show all of the cards and will remove multiple colors.
Work this past week - Made modifications as suggested the previous week. Input was taken from Akshay and Shashank (GA’s).
Work next week - Modifications for scorecard mock-ups
Open Discussions/Questions
Reviewed current scorecard mock-ups and changes to be done and shown at next week's meeting. Won’t be able to cookie-cut all scorecards because data is different, so some of these mock-ups will change.
Valeria had questions about the goals. Faculty awards tab, only allowed percentages instead of numbers, we may need to add numbers. Valeria will forward the email to Sangita.
Valeria suggested having a link to the university’s goals with a OneUSF version of the scorecard. OneUSF is per metric. Shruti also likes the OneUSF scorecard per metric and it will align better when we do the Point-In-Time metrics.
Sangita asked if we should have a prior year button on these scorecards. Do we do this now or do we wait until we have the Point-In-Time dashboards?
Valeria suggested that we have space for 2 buttons, one now to go back to the 5-year trend and another button for the prior year that will eventually be the point in time.
Jay suggested that the team look at the mobile view template in PowerBI.
8/31/2023 Status Meeting Agenda/Notes
Agree that the Scorecard Metrics are the following: CONFIRMED by Shruti and Valeria
FTIC Freshman Retention Rate: USF, Same College
Academic Progress Rate (2nd Year Retention with GPA above 2.0): USF, Same College
FTIC 4-Year Graduation Rate: USF, Same College
FTIC 4-Year Graduation Rate for Pell
FTIC 6-Year Graduation Rate: USF, Same College
FTIC 6-Year Graduation Rate for Pell
FCS AA Transfer 3-Year Graduation Rates: USF, Same College
Total Research Expenditures
Faculty Awards
Total Research Expenditures per Tenure Track FTE (Don’t have goals for this)
Citation Impact (upcoming)
Annual Fundraising (upcoming)
Agree on Schedule (@Shruti Kumar to get back to @Sarah Thomas and @Jayakumar Unnikrishnan to confirm)
Design and development of College Scorecards (Aug 18 - Sept 29)Website modifications (Oct 2 - 11)
QA and approval of Scorecards (Oct 9 - 13)
Go Live for College Scorecards (Oct 20)
Review work done on Scorecards - Recreate Mocks in PowerBI, a few versions to get feedback and will review newer version next week
Work this past week - Mockup in PowerBI
Work next week - Incorporate feedback and reshow mock-up in PowerBI next week. No work until mock-ups are approved.
Open Discussions/Questions - Discussed different views shown (6 total). Shruti took detailed notes on what needs to be changed for mock ups.
8/24/2023 Status Meeting Agenda/Notes - Meeting cancelled, Grad Asst.s have not had much time to work on this, so no status updates and no questions.
8/14/2023 Status Meeting Agenda/Notes
Review College Scorecard design that Sangita has drafted
Design 1 and Design 2 were shown. Valeria preferred Design 1, but said that Design 2 could be used for a printable version.
Valeria asked to have a filter added at the campus level.
Review Project Watch List - N/A