Jump to Meeting Agenda and Notes
Project Status
Phase 2 - Design and creation of College Scorecards. COMPLETED
Project Timeline
Last updated and approved by Valeria on 09.07.2023.
Mock-up and approval of College Scorecards (Aug 18 - Sept 22) Completed
Website modifications and approval and CMS modificaitons (Nov 9 - Nov 14) Completed
Development and QA of Scorecards (Sep 20 - Nov 9) Completed
Go Live for College Scorecards (Nov 14 - 15) Completed and went live on Nov. 17 at 1pm
Project Watch List
Citation Impact for Phase 1 dashboard and Phase 2 scorecard. No way to capture this at the college level, but Jay is working with Strategic Metrics to get this information.
Technical documentation (Action Item from Retrospective) - Not specific for phase 2, keep it as a separate issue.
These three metrics will not be available until March, so no need to discuss until then.
Phase 1 and 2 - Non-Medical Science & Engineering Research Expenditures ($M) - Not available until after 3/1/24
CA Phase 1 and 2 - Science & Engineering Research Expenditures ($M) - Not available until after 3/1/24
CA Phase 1 and 2 - Effective F&A Rate (%): By College (Effective F&A Rate) - Not available until after 3/1/24
Meeting in March to start discussion on Phase 3 scope and timeline.Later phase - migration to Microsoft Fabric for Point in Time Dashboards
Meeting Agenda and Notes (most recent on top)
1/18/2024 Status Meeting Agenda/Notes
Review Watch List - Next week, it was just reviewed on 1/11/2024 meeting.
Open Discussion
Sangita showed scorecards to confirm Total Fundraising has been moved. Valeria will send the Deans an email to let them know that Total Fundraising is now available.
Sangita would prefer not to do an asterisk on the year for incomplete data 2023-24, so that a difference in data is explainable and noted; not all data will have been collected yet. However, she will put a note on the table/scorecard/graph/window so that it is noted.
Faculty Awards - Sangita was asked to break out TARU, NRC, etc. are broken out into individual items, not TARU and then TARU/NRC… in the drop down for Faculty Awards.
Retention needs a detailed discussion, Shruti, Valeria, Glendalis, Steve, Jay, Leslie and Kim on this meeting. Shruti will have Tammy schedule.
1/11/2024 Status Meeting Agenda/Notes
Standardize Metrics Names between projects, including College Accountability. Listed as a separate project, but will start soon. Sarah, Shruti and Valeria will discuss.
Citation Impact for Phase 1 dashboard and Phase 2 scorecard. No way to capture this at the college level.
Total Fundraising - Meeting was completed with Susan Vandermast & Noreen Seagrest. They have agreed to provide the data but have not heard back. Valeria will follow up. They have fundraising data now.
Documentation
Refresh schedule complete.
Received information on Invention Disclosures?
Technical documentation (Action Item from Retrospective) - Not for phase 2, but add it as an issue, but not specific for phase 2, keep it as a separate issue.
These three metrics will not be available until March, so no need to discuss until then.
Phase 1 and 2 - Non-Medical Science & Engineering Research Expenditures ($M) - Not available until after 3/1/24
CA Phase 1 and 2 - Science & Engineering Research Expenditures ($M) - Not available until after 3/1/24
CA Phase 1 and 2 - Effective F&A Rate (%): By College (Effective F&A Rate) - Not available until after 3/1/24
Further meetings to track these items above or hold until March? Keep these as is and at the same time. If we do not need we will cancel.
Meeting in March to start discussion on Phase 3 scope and timeline.
12/14/2023 Status Meeting Agenda/Notes
Meeting repurposed for Retrospective. See separate link for retrospective notes in project dashboard.
12/05/2023 Status Meeting Agenda/Notes
Retrospective for Phase 2? Yes
Address Watch List:
Standardize metric names - ask Valeria across project/dashboards - Yes, we need to do this.
Documentation for College Accountability - Working on refresh schedule, should be done for holidays. Need to get information on Invention Disclosures.
CA Phase 1 - Fundraising - yes (Total Fundraising) - DONE
CA Phase 1 - Public University National Ranking - Do not have at the college level - No work needed
CA Phase 1 - Citation Impact - (See below)
CA Phase 1 - Non-Medical Science & Engineering Research Expenditures ($M) - Not available until after 3/1/24
CA Phase 1 - Science & Engineering Research Expenditures ($M) - Not available until after 3/1/24
CA Phase 1 - Overall F&A Rate (%): By College (Effective F&A Rate) - Name change is done, however, data is not available until after 3/1/24
Follow up from last meeting
Citation Impact (upcoming) - Timeline? Waiting on a different group to get information
Annual Fundraising (upcoming) - Was meeting set up and/or held with Valeria, Susan Vandermast & Noreen Seagrest? Meeting was done. Timeline? Valeria/Jay said that they have agreed to provide the data but have not heard back. Valeria will follow up.
Hide Effective F&A Rates page - Deena in Research will get back to us with updates and let us know when we can “unhide” this page. Not before 11/15. Timeline? (See above)
Deena to give information on discrepancies to be able to explain and Sangita will add this to online documentation. Not before 11/15. Timeline? DONE
Faculty awards will be distributed when it is available; need to establish a goal for total awards. If TARU is around for next year, then we can set a goal for this subset of awards. Keep on watch list or establish timeline? DONE
Faculty Awards is being updated manually - Do we plan on automating this or is this good for now? Keep this manual, no work needed.
11/06/2023 Status Meeting Agenda/Notes
Meeting is to allow Q&A for Sangita and her team.
11/02/2023 Status Meeting Agenda/Notes
Project Schedule
Mock-up and approval of College Scorecards (Aug 18 - Sept 22) Complete
Website modifications and approval (Oct 2 - Oct 20) Hennadii - On Track to start
Development and QA of Scorecards (Sep 20 - Nov 13) Sangita - On Track
Go Live for College Scorecards (Nov 14 - 15) - Team - On Track
Discussions
Watch List
Valeria will resend the email to Susan Vandermast & Noreen Seagrest for Annual Fundraising metric and goal. Meeting is going to be set up.
Grant dollars and Research Expenditures, Grant proposals submitted.
Hide Effective R&A Rates page - Deena in Research will get back to us with updates and let us know when we can “unhide” this page. Not before 11/15
Deena to give information on discrepancies to be able to explain and Sangita will add this to online documentation. Not before 11/15.
Documentation online (including refresh dates). Refresh calendar updated in Confluence, updates needed?
Have all changes for the scorecard format been completed? Done.
On Track for Nov. 7th. Not including Hennadii’s work which is a quick change. Sarah’s CMS change is also fast. Still on Track? No, changed final due date to Nov. 13th. 8 out of 12 metrics are done for scorecards. All colleges are done for each metric.
The CA reports completed during phase 1 will be refreshed. 80% are done. Research is noted above with data still changing. Complete by Monday, 11/6. Dependent of Fall preliminary bench.
Work this past week
Scorecards and the total research expenditures and refresh for the metrics for phase 1.
Work next week
Complete scorecards by Nov. 13th. Go live the week of Nov. 13-17th.
Open Discussions/Questions
CMS web site modifications
Sarah to create a Box folder to share screen shots and send out link.
Let everyone know that Advancement awards will be distributed when it is available. Valeria will send this information. Need to establish a goal for total awards. If TARU is around for next year, then we can set a goal for this subset of awards.
Show Total Awards Goal and for OneUSF (sum of all the colleges).
Tenure track FTE is during a fiscal year. Numerator and denominator will both change. No goal as it is a completely new metric.
Shruti shared the document she has created for the refresh schedule. Currently in a spreadsheet, but she can transfer to another format.
Two sets of scorecards, one for the colleges and the other for One USF.
The faculty awards metrics is being updated manually right now.
Give screenshots to Deans on Monday 11/13
Go live week of 11/13-17
10/26/2023 Status Meeting Agenda/Notes
Project Schedule
Mock-up and approval of College Scorecards (Aug 18 - Sept 22) Complete
Website modifications and approval (Oct 2 - Oct 20) Hennadii - On Track to start
Development and QA of Scorecards (Sep 20 - Nov 7) Sangita - On Track
Go Live for College Scorecards (Nov 14 - 15) - Team - On Track
Discussions
Watch List: Valeria will resend the email to Susan Vandermast & Noreen Seagrest for Annual Fundraising metric and goal. Meeting is going to be set up.
Watch List: Grant dollars and Research Expenditures, Grant proposals submitted.
Goals (Sangita)
Some of the goals are blank, Valeria will follow up and has sent an email/note to EDU and MAR. SCI regarding these empty goals. DONE
Hide Effective R&A Rates page - Deena in Research will get back to us with updates and let us know when we can “unhide” this page. UPDATES? Not before 11/15
Deena to give information on discrepancies to be able to explain and Sangita will add this to online documentation. Status - Not before 11/15.
Sangita is tracking what data is changed when and will put into the CA Team meeting chat and tag Valeria. Have all changes for the format discussed last week been implemented and approved? Working on it now.
Valeria said we need to give the Colleges a few days to preview the Scorecards before they go live. WE NEED TO HAVE ALL DONE BY NOV. 9th, so that Valeria can send out to Colleges to preview.
On Oct. 13, Sangita said that the Scorecards should be done in 3 weeks, by Nov. 3rd. On Track for Nov. 7th. Not including Hennadii’s work, quick fix.
The CA reports completed during phase 1 will be refreshed. 80% are done. Research is noted above with data still changing. Status?
Refresh Documenation (Timeline?)
Sangita explained that she is putting refresh dates with the data and will demo this in the future.
Shruti showed everyone the Confluence page that has the refresh information. Kevin is updated and should be done when?
Work this past week
Scorecards and the total research expenditures and refresh for the metrics for phase 1.
Work next week
Scorecards and the total research expenditures and refresh for the metrics for phase 1.
Open Discussions/Questions
10/19/2023 Status Meeting Agenda/Notes - Meeting cancelled.
10/13/2023 Status Meeting Agenda/Notes
Project Schedule
Mock-up and approval of College Scorecards (Aug 18 - Sept 22) Complete
Website modifications and approval (Oct 2 - Oct 20) Hennadii - On Track to start
Development and QA of Scorecards (Sep 20 - Nov 9) Sangita - On Track
Go Live for College Scorecards (Nov 14 - 15) - Team - On Track
Discussions
Hennadii - not a large amount of work and can be done towards the end of the project.
Valeria will resend the email to Susan Vandermast & Noreen Seagrest for Annual Fundraising metric and goal. Meeting is going to be set up. Valeria asked to keep this on the watch list.
Goals (Sangita)
Some goals are empty - What do we do with these? Sangita will be filling these in according to previous year, etc. (same technique that has been done in the past.
Would like to verify goals with colleges - FTIC retention at 93%, only BCS has FTIC Ret at 93%. This will be done with the colleges according to Valeria. Some of the goals are blank, Valeria will follow up and has sent an email/note to EDU and MAR. SCI regarding these empty goals.
Hide Effective R&A Rates page - Sangita was directed to do this by Deena/Research.
Sarah to park this issue and revisit. Grant dollars and Research Expenditures, Grant proposals submitted.
Deena to give information on discrepancies to be able to explain and Sangita will add this to online documentation.
Sangita is tracking what data is changed when and will put into the CA Team meeting chat and tag Valeria.
Valeria said we need to give the Colleges a few days to preview the Scorecards before they go live.
Research metrics
Access issue with PowerBI dataset, no one can see data. Sangita working with IT and Deena to get this fixed. Resolved.
Research metrics give a “gateway error” every time it is refreshed. IT is working to resolve. Resolved.
The CA reports completed during phase 1 will be refreshed. 80% are done. Research is noted above with data still changing.
Refresh Documenation (wait until Next Week)
Sangita explained that she is putting refresh dates with the data and will demo this in the future.
Shruti showed everyone the Confluence page that has the refresh information. Kevin is updated and should be done when?
Work this past week
Scorecards with Research and Endowment
Work next week
Scorecards should be done in 3 weeks.
Open Discussions/Questions
Valeria asked to bring up the mock to quickly review.
10/05/2023 Status Meeting Agenda/Notes - Meeting Cancelled No Updates
09/28/2023 Status Meeting Agenda/Notes
Project Schedule
Mock-up and approval of College Scorecards (Aug 18 - Sept 22) Complete
Website modifications and approval (Oct 2 - Oct 20) Hennadii will take on responsiblity (Chirag can assist if needed)
Development and QA of Scorecards (Sep 20 - Nov 9) On track
Go Live for College Scorecards (Nov 14 - 15)
Discussion
Valeria will resend the email to Susan Vandermast & Noreen Seagrest for Annual Fundraising metric and goal. Meeting is going to be set up. Valeria asked to keep this on the watch list.
Modifications for scorecard mock-ups. None needed all scorecard mockups have been approved.
Grant dollars awarded will be connected to the IT central powerbi dataset for pulling in the numbers. Any follow up needed?
Remaining CA reports which are manually updated will be refreshed this week. The CA reports completed during phase 1 will be refreshed.
Work this past week
Working on College Scorecards
Work next week
Continue working on College Scorecards
Open Discussions/Questions
Build out a timeline document similar to the PPA version and add it to the CA website. Timelines need to include refresh cycles etc. Potentially look at push notifications to let colleges know. Sangita explained that she is putting refresh information with the data and Shruti showed everyone the Confluence page that has the refresh information. Sangita will show the team an example of the documentation in the future.
09/20/2023 Status Meeting Agenda/Notes
Project Schedule
Mock-up and approval of College Scorecards (Aug 18 - Sept 22)
Website modifications and approval (Oct 2 - Oct 20)
Development and QA of Scorecards (Sep 20 - Nov 9)
Go Live for College Scorecards (Nov 14 - 15)
Review work done on updated mock-up of Scorecards
Decisions we made last meeting. No need to review progress, just as an FYI for the team.
Separate tab for the “OneUSF” grouping of metrics.
The plan is to add a single link called “Scorecards” on the main College Accountability landing page.
Work this past week
Link the Faculty Awards page (Taru report) on the research website as documentation from ODS’s site has been done.
Modifications for scorecard mock-ups
Research metric of “Overall F&A rate” has been modified to “Effective F&A Rate” based on Deena’s suggestion.
Work next week
Grant dollars awarded will be connected to the IT central powerbi dataset for pulling in the numbers.
Remaining CA reports which are manually updated will be refreshed this week.
Open Discussions/Questions
Build out a timeline document similar to the PPA version and add it to the CA website. Scorecards will replace PPA. Need to figure out where the scorecards will appear on the CA website. Timelines need to include refresh cycles etc. Potentially look at push notifications to let colleges know.
Valeria settled on the all consistent green color for the scorecard for now.
Goals for TARU & Others metric was not asked for from Research and will need to be accommodated in the scorecard.
Valeria will resend the email to Susan Vandermast & Noreen Seagrest for Annual Fundraising metric and goal.
In October for the first release there will be no “Previous Year” button.
09/14/2023 Status Meeting Agenda/Notes
Project Schedule
Mock-up and approval of College Scorecards (Aug 18 - Sept 22) - On track
Website modifications and approval (Oct 2 - Oct 20)
Development and QA of Scorecards (Sep 20 - Nov 9)
Go Live for College Scorecards (Nov 14 - 15)
Review work done on updated mock-up of Scorecards
Faculty awards tab change from percentages to numbers - Progress?
Decisions we made last meeting. No need to review progress, just as an FYI for the team.
Note that we will add OneUSF scorecard
Note that we will have space for 2 buttons, one now to go back to the 5-year trend and another button for the prior year that will eventually be the point in time.
Note that we need to look at the mobile view template in PowerBI
Work this past week
Received feedback from Valeria, Jay and SSRA built mockup showing scorecard with the feedback in place.
Work next week
Modifications for scorecard mock-ups
Open Discussions/Questions
Valeria pointed out a need for ADA compliance and color schemes. Sangita is reviewing it and explore the options. Not a priority but would recommend doing a due diligence on ODS’s side. Shruti volunteered to work with Stephanie to find out more about it. Review by next meeting.
On mobile, the view will show one card at a time as per Sangita.
Discussion on whether to go for 6x2 format (portrait) or 2x6 (landscape) format will be based on how it appears when printed on the page. Preference is portrait mode (6x2). Will have done by next meeting.
Groupings by color was discussed and Valeria was not sure if we needed it. Perhaps having groups of colors by domain made more sense. SSRA will built mockups showing the options next week
Faculty awards is counts and no percentages. Current year was done as pct by mistake. This has been fixed and accounted for in next year’s cycle. For this year, Valeria has already explained this to the colleges.
The plan is to add a single link called “Scorecards” on the main College Accountability landing page.
Separate tab for the “OneUSF” grouping of metrics.
Academic Affairs and Health breakouts may be needed. For certain metrics this grouping may be a challenge and would need to be investigated (e.g. Others). Valeria explained how the Provost would like to see this distribution especially on items like Research related metrics. SSRA will investigate and come back next week with recommendations.
Research dataset will be online and pulled in starting next week. Last update (done manually was back in July) but now will be automatically pulled in.
Valeria suggested to link the Faculty Awards page (Taru report) on the research website as documentation from ODS’s site.
09/07/2023 Status Meeting Agenda/Notes (Shruti provided a new schedule below)
Agree on Schedule (Original Proposal)
Design and development of College Scorecards (Aug 18 - Sept 29)
Website modifications (Oct 2 - 11)
QA and approval of Scorecards (Oct 9 - 13)
Go Live for College Scorecards (Oct 20)
Proposed revised schedule from Shruti. This schedule was approved by Valeria
Mock-up and approval of College Scorecards (Aug 18 - Sept 22)
Website modifications and approval (Oct 2 - Oct 20)
Development and QA of Scorecards (Sep 20 - Nov 9)
Go Live for College Scorecards (Nov 14 - 15)
Review work done on updated mock-up of Scorecards - Must be able to show all of the cards and will remove multiple colors.
Work this past week - Made modifications as suggested the previous week. Input was taken from Akshay and Shashank (GA’s).
Work next week - Modifications for scorecard mock-ups
Open Discussions/Questions
Reviewed current scorecard mock-ups and changes to be done and shown at next week's meeting. Won’t be able to cookie-cut all scorecards because data is different, so some of these mock-ups will change.
Valeria had questions about the goals. Faculty awards tab, only allowed percentages instead of numbers, we may need to add numbers. Valeria will forward the email to Sangita.
Valeria suggested having a link to the university’s goals with a OneUSF version of the scorecard. OneUSF is per metric. Shruti also likes the OneUSF scorecard per metric and it will align better when we do the Point-In-Time metrics.
Sangita asked if we should have a prior year button on these scorecards. Do we do this now or do we wait until we have the Point-In-Time dashboards?
Valeria suggested that we have space for 2 buttons, one now to go back to the 5-year trend and another button for the prior year that will eventually be the point in time.
Jay suggested that the team look at the mobile view template in PowerBI.
8/31/2023 Status Meeting Agenda/Notes
Agree that the Scorecard Metrics are the following: CONFIRMED by Shruti and Valeria
FTIC Freshman Retention Rate: USF, Same College
Academic Progress Rate (2nd Year Retention with GPA above 2.0): USF, Same College
FTIC 4-Year Graduation Rate: USF, Same College
FTIC 4-Year Graduation Rate for Pell
FTIC 6-Year Graduation Rate: USF, Same College
FTIC 6-Year Graduation Rate for Pell
FCS AA Transfer 3-Year Graduation Rates: USF, Same College
Total Research Expenditures
Faculty Awards
Total Research Expenditures per Tenure Track FTE (Don’t have goals for this)
Citation Impact (upcoming)
Annual Fundraising (upcoming)
Agree on Schedule (Shruti Kumar to get back to Sarah Thomas and Jayakumar Unnikrishnan to confirm)
Design and development of College Scorecards (Aug 18 - Sept 29)Website modifications (Oct 2 - 11)
QA and approval of Scorecards (Oct 9 - 13)
Go Live for College Scorecards (Oct 20)
Review work done on Scorecards - Recreate Mocks in PowerBI, a few versions to get feedback and will review newer version next week
Work this past week - Mockup in PowerBI
Work next week - Incorporate feedback and reshow mock-up in PowerBI next week. No work until mock-ups are approved.
Open Discussions/Questions - Discussed different views shown (6 total). Shruti took detailed notes on what needs to be changed for mock ups.
8/24/2023 Status Meeting Agenda/Notes - Meeting cancelled, Grad Asst.s have not had much time to work on this, so no status updates and no questions.
8/14/2023 Status Meeting Agenda/Notes
Review College Scorecard design that Sangita has drafted
Design 1 and Design 2 were shown. Valeria preferred Design 1, but said that Design 2 could be used for a printable version.
Valeria asked to have a filter added at the campus level.
Review Project Watch List - N/A
Work this past week - Draft of College Scorecard design.
Work next week - GA’s and Hennadii under Xiaoying’s guidance will work on college scorecards.
Open Discussion and Questions
Sangita will be out Aug. 16th - Sept. 6th and Xiaoying will be supervising her GAs.
Citations - We are not planning on doing Citations as Jay noted that we do not have this data at the college level.
Total Research Expenditures - This metric will not move much. While Sangita is out Chirag will have to do trainings on the research dataset. Add the faculty FTE data and there is research data that doe not match 100%, this will have to be researched. Valeria added that this may need to be discussed on whether FDRA should own this metric as it is a faculty metric (calculation of metric, not display)
Annual Fundraising - getting this data will be delayed.
Remaining metrics will move forward in Sangita’s absence.
8/11/2023 Slalom Meeting Notes
No formal agenda, so all topics were points of discussion during the meeting.
Attendees
Slalom: Brian Beesley, Sergio Martinez, Michelle McNulty, Prasanth Sistla
USF: Sarah Thomas, Jay Unnikrishnan, Shruti Kumar, Sangita Singh, Chirag Jain, Hennadii Balashov, Xiaoying Liu, (GAs attending were Shashank and Akshay)
Slalom suggested that if the data sources have historical data, then they recommend using data flows that feed PowerBI data sets.
They asked if SAS EG has the ability to connect to Data Lake Storage (Azure). It does.
The suggested to use Forms or Survey Monkey or any data collecting software to gather data from persons who have to provide data. Microsoft Forms allows for back and forth with users during the goal definition process.
Currently, Microsoft Forms is free, not sure about the future cost. However, PowerApp, which is also needed is not free.
Prasanth suggested that we consider using a pipeline to pull data from different data sources and store this in data lakes. We can use VNet gateways to pull data from the cloud.
USF/ODS will provide Microsoft Rep contact information to Slalom and they can pass along their suggestions to Microsoft (give them information to work with USF IT and ODS).
8/10/2023 Status Meeting Agenda/Notes
Decisions conveyed to Slalom - For the short term, the project leads are planning to utilize option 1 with P drives, but would like a longer term plan to move towards using Microsoft Fabric. We would like a plan which would help to lead the project team from option 1 to an eventual use of option 3 (possibly show an optimal phased approach). We are also looking to Slalom to help provide the “why” on eventually moving to option 3 to help pitch the idea to leadership.
Plans for Analysis and initial design - Shift to doing scorecards first, then point-in-time dashboards. If for some reason scorecards cannot be finished, plan B would be to create the pdfs. However, these pdfs could be screenshots or downloads. Will need to filter by college
Status on Report from Slalom - Need to find out when they will provide their final report.
Review Project Watch List - N/A
Work this past week - N/A
Work next week - Sangita to design the college scorecards and present to Valeria on Monday 8/14 for approval.
Open Discussion -
Question on Faculty Awards - the flag needs data to be updated, Victoria Stewart is updating this.
Sangita will be out Aug. 16th - Sept. 6th.
7/26/2023 Status Meeting Agenda/Notes
Debrief from Slalom Workshop
Hennadii provided a synopsis of three potential solutions that Slalom provided.
1. use p drives or database
2. move away from SAS query directly from data sources.
3. Use Microsoft Fabric, may not be immediately feasible. So, the ODS team needs to decide on option 1 or 2.
Sangita said IT is looking into data lakes, but we do not know the effort/cost to convert into data flows.
Shruti discussed different options and different Microsoft toolsets and got IT’s perspective.
We are thinking of starting with our SAS code, modifying code and breaking it up to create point-in-time metrics.
We need to decide if we store the master data set in a data store or do we keep it on the P drive.
We can automate by using SAS scheduler to some extent, but it will not be a fully automated solution.
Want to get a SAS server account, but the cost was over $10K to get a SAS server license.
Another option would be to store our master data set in a database. This would also involve getting a server which would be a cost.
IT is planning to provide data lakes but that will not be completed in time for this project. IT said we could work together for a future solution.
We want to standardize source code control.
We want a data store, but it is not a show stopper now. We would use a data store by bringing data into SID, a data lake, etc.
Valeria said we need a broader conversation on ODS data needs as a whole. We need to determine what our needs are now and would the future path look like. She would like us to pause and have a broader conversation with IT and discuss funding and create a larger budget ask.
Jay said that Slalom will provide a document that will help us with our discussions with IT. If you want a zero-dollar solution until we are ready to move to the final state. He also hesitates to use the existing state, he wants to use an intermediate state that will prep us for the final state. He wants to use option 2 to move away from SAS to create data flows which would be zero dollars and it would be more easily converted to pulling from a data lake. Option 2 provides zero cost and prepares for the future (Jay).
Sangita says this is not zero cost, there will be a cost to convert code from SAS which will take resources and time.
Valeria agreed that we cannot keep status quo. SAP will go away next year, contract will stop on December 2024. We need to know how this factors into our decision and plan.
Shruti - maybe we can have conversations with IT with data flows, since our code is already in SAS, she is inclined to want a database. We can add to an existing database.
Slalom and Jay proposed a design which were similar. Jay wanted to start converting to data flows, maybe some of them. Data flow is to create a data set for others to use for a single use. Data flows are more useful if users do not know table structure. For developers, we could use a data lake to access data.
Sangita says that we have a tight timeline and we do not likely have the time to convert to data flows.
Shruti says we cannot wait to bring team up to speed and to convert to data flows. She wants to do the ETL in SAS and convert at a later time. Need to come up with plans and time. Jay agreed to this plan and Sarah will draft and send.
Plans for Analysis and initial design - This will be done after the plan with Slalom is defined.
Status on Report from Slalom - Due Date - Next time we meet with Slalom, we can get this date.
Review Project Watch List - Not discussed.
Work this past week - Not discussed.
Work next week - Not discussed.
Open Discussion
Fundraising Data - Sangita said that we would like to have some movement on the Fundraising data. She asked if Valeria would mind following up with Susan to get the needed information.
Research Data - Sangita noted that the summation of the goals at college level will not match up with Accountability Plans totals. This is because of timing. Sangita talked to Dina and she said they have no way to deliver data at an earlier time frame. Valeria said we may need to have a conversation with Dina to confirm.
7/19/2023 Status Meeting Cancelled
7/12/2023 Status Meeting Agenda/Notes
Slalom Workshop July 18, 11-3pm
Slalom Deliverables
Access - Jay - FERPA - Done
Box information was shared - Sangita has sent all of the following information in the spreadsheet.
Proc SQL
Data Flow Diagram
Screenshots of previous PPA
Status on Current Work
Documentation style shift - Valeria has given the ok to move forward but this is low priority.
Reviewing phase 1 dashboards - to be more user friendly - being done in the background - Sangita
Research data (Federal Expenditures and Total Expenditures) was available last week. Status - Sangita
Work this past week - Done with Slalom’s deliverables and put into the folder. Add a few agenda items and send to them - Sangita. Were supposed to work on the goals, and they are 99.9% done. Valeria will be sending to leadership, goal work done by Xiaoying. Refreshed metrics for historical trends. Button that we needed to add to the image has been completed. Dina is supposed to send this week, Sangita will follow up with her.
Work next week - Workshop.
Open Discussion -
SSRA is not working on Endowment which is to be updated in September. We won’t have goals for Endowment, but will have goals for Annual Fundraising. Valeria will follow up on last email thread regarding the delay of goals for Endowment.
Valeria observed that they used to see goals per tab and she will convey this.
Reviewed and shored up the CA Refresh Schedule.
Next week’s meeting will be canceled.
7/5/2023 Status Meeting Agenda
Slalom Workshop July 18, 11-3pm - Did everyone get invitation for meeting?
Slalom Deliverables
Access - Jay - FERPA 2 pending (Swapna and Gabriella)
Box information will not be shared until tomorrow - Sangita
Spreadsheet with details of metrics and sources - Done - Sangita
Proc SQL - Sangita - Sample code for Retention and Graduation - Sangita will send
Data Flow Diagram - Done
Screenshots of previous PPA - Done
Access to Box - Done - Sarah
Status on Current Work
Need to discuss documentation, Sangita wants to change style - Sangita suggested that the documentation should be acutally in the report instead of a separate place. Jay said when documentation is presented, it is tied to the report and updated documentation will have to be published with an updated data report. Valeria likes this but acknowedges the struggles that Jay has pointed out. She suggested a “hover-over” pop up text with information. Shruti likes the page, and thinks this needs to be documented and wants to keep the technical documentaton. For related documentation, we can add a link at the bottom of the page to provide more information. Slowly progress towards this and Sangita needs to ensure that the maintenance is not too heavy. Valeria says this is lower priority.
Reviewing phase 1 dashboards - to be more user friendly - being done in the background - Sangita
Working on Research metrics with Deena - Wants to change the name of “Grants Awarded” instead use “Grant Dollars Awarded” - Valeria approved
Research data (Federal Expenditures and Total Expenditures) will be available from IT this week. Sangita still expects this by next Tuesdsay, Deena will update. Instructor lead training session in August and is encouraging SSRA/Other ODS teams to sign up and attend. Valeria wants others in ODS to take this course also.
College Accountability - Need to change “Expenditures” to “Research Metrics” - Valeria approved
We do have Endowment data even down to the department level. Endowment is a part of fundraising.
Work this past week - Slalom Work, Research Data and Documentation in the background.
Work next week - Slalom completion, Research Data work and Phase 2 metrics work. Working on how to automate.
Open Discussion
Valeria likes the view that Sangita displayed for the scorecard. We will show the current year and previous year? Valeria said yes, we do need current year and previous year at the same time.
Need to have Goal Setting discussion. Templates need to be set up and updated. We need to have goal projections. Xiaoying will start these templates and will send to Valeria and Shruti for review and approval.
6/28/2023 Status Meeting Agenda and Notes
Slalom Workshop July 18, 11-3pm
Slalom Deliverables
Access - Jay - FERPA for Brian and Michelle is done, Swapna is in progress, Gabriella not yet
Spreadsheet with details of metrics and sources
Proc SQL
Data Flow Diagram
Screenshots of previous PPA
Challenges in writing
Next priorities
Create a shared Box folder with Slalom
Forward Slalom workshop invitation to SSRA
Working on Slalom deliverables
Work this past week, next week
Analysis of phase 2
Endowment metric for phase 1 - in review
Upgrading Phase 1 metrics
Open Discussion
Need to discuss documentation, Sangita wants to change style
Phase 1 metrics some have been refreshed (UG Research, Service Learning)
Reviewing phase 1 dashboards to be more user friendly
Working on Research metrics with Deena - Wants to change the name of “Grants Awarded” instead use “Grant Dollars Awarded” - Ask Valeria next week
Research data (Federal Expenditures and Total Expenditures) will be available from IT next week.
College Accountability - Need to change “Expenditures” to “Research Metrics” - Ask Valeria next week
6/21/2023 Status Meeting Agenda and Notes
Slalom Status
Update on request to move to July 17 - No updates, assume July 10 is still the date
10 Metrics and 5 Data Sources - Shruti/Sangita - Showed metrics and sources spreadsheet
Retention and Graduation Metrics Deep Dive - Sangita - In progress
Update on Slalom access - Jay - Not all Slalom resources have completed FERPA
Screen shots of current dashboards for 10 metrics - Shruti/Hennadii - In Progress
Screen contents for phase 2 and phase 3 (Shruti planning on showing PPA to Slalom) - Shruti - Done
Metrics added to Jira for phase 2 - Sarah - Done
Review Project Watch List - Sarah - Not done, holding until Slalom work is done.
Work done last week, Work to be completed next - Sarah/Team - Slalom work last week, continuing next week
6/13/2023 Status Meeting Agenda and Notes
Review data refresh schedule/spreadsheet - Sangita has shown the spreadsheet with the data refresh schedule for 5-Year and Progress-to-Goal metrics.
Slalom update
June 13th - Initial meeting with Slalom
Request to move to July 17 is up to Slalom
Choose 10 metrics and 5 data sources - Due June 20 - Sangita/Shruti
Deep dive for retention and graduation metrics - Due June 20 - Sangita/Shruti
Deep dive for ALL Metrics due by June 30th - Sangita
Screenshots of existing dashboards of 5 Year Trends for Slalom retention/grad and PPA - Shruti, Sangita/Hennadii will look into other metrics for sending to Slalom - Next week
Dashboard design for phase 2 and phase 3 - Shruti/Sangita
Standardizing Metric Names - Let’s postpone this until after Slalom work is done. - Agreed
What do we need to do with the following metrics (if anything)
Public University National Ranking (Upcoming) - This is a metric that is on the phase 1 dashboards, but is listed as TBD. We will keep this on the watch list.
Citation Impact (Upcoming) - Jay will get back to Sangita on this
Following Metrics have been updated in the Production (The trend is by clicking on the image, The College filter on the filter panel is showing only applicable colleges and not all colleges) - Sangita - Did not have time to review this; postponed until after Slalom deliverables have been completed.
Priorities
Slalom deep dive retention and graduation due by June 20 - Sangita
Slalom deliverables all due by June 30th - Sangita/
Slalom access - due by June 30th - Jay
Work on standardizing metric names - Sangita
Compile list of items to finish from Phase 1 - Sangita/Shruti
Discuss to stop working on Phase 1 items until Slalom deliverables are complete.
Last week work, next week work
6/7/2023 Status Meeting Agenda and Notes
Jira epics added from last meeting. Done. Sarah
Slalom update - Introductory meeting on calendar for June 13. Date for workshop is TBD. Sarah
Review metrics for phase 2 and phase 3, see Project Decisions for 5/31.
Deliverables for Slalom - Status - Due ___ - Shruti/Sangita
Determine maximum 10 metrics for Slalom to work with
Determine which data sources these metrics come from, cannot exceed 5 data sources
Data refresh schedule for Slalom
Schedule to update data refreshes, matrix to be created - Status 50% - Due first pass on 6/14 - Sangita for phase 1. VG suggested we address phase 2 and 3 data refreshes into the matrix. Sangita will add the metrics for phase 2 and 3 and review what she has done for next week.
Do we need to review Research metrics from phase 1 (any progress)? No progress, Jay said data set is almost complete. Sangita will follow up with Dina on Friday. We don’t want to miss the chance to make changes or suggestions. Jay suggested Sangita join the next Sprint Review session tomorrow.
Review updates to dashboard - Sangita/Akshay - Same color for hover color and gray for buttons with beige for hover color.
Faculty Awards, Fellowships and Memberships - Faculty Awards done once a calendar year, this should remain once a year. This isn’t refreshed more than annually, it still needs to be in the same place as other Phase 2 metrics so that users can see data in the same format as the other metrics in this list. Faculty awards do have goals. These need to show up on phase 3 College Scorecards.
5/31/2023 College Accountability Phase 2 - Status Meeting Agenda and Notes
Show Jira plan and CA board with items moved to phase 2. Done
Update on Slalom contract - Sarah Thomas
Deliverables for Slalom - Still in progress
Determine maximum 10 metrics for Slalom to work with
Determine which data sources these metrics come from, cannot exceed 5 data sources
Determine which systems that Slalom consultants will need access to
Determine who are the SMEs for these metrics and data sources
Set up for Slalom (netid, global protect, vpn, and access to systems) - Need someone to coordinate - Jay volunteered for this.
Review first draft of schedule that Sangita has set up. This may not be complete, but it will give her an opportunity to ask questions and get feedback. Still in progress.
Sarah to add the following tasks to Jira (groups approval). Reviewed with meeting attendees and assignments made.
Slalom information set up (Shruti)
Slalom access (Jay)
Slalom workshop scheduling on Teams Sarah Thomas Meeting invitation sent by Slalom.
Data Model Analysis and Design (Sangita)
Data Model Implementation (TBD)
Metrics to include design of report and SQL for data pull (Sangita/Hennadii/Xiaoying) for 5 Year Trends - this will be for all metrics that we included for Phase 1 - ACTUALLY NO, We reviewed a list of metrics that should be included. See Project Decisions for 5/31.
Website design changes (Shruti)
Website modifications (Chirag)
Proposed Timeline to review - Not ready to review
Prepare for Slalom Workshop (June 12- June 30)
Slalom Workshop - (July 10 - 24)
Analysis and internal assessment of Slalom Design (July 24 - Aug 4)
Development of Data Model (Aug 7 - Aug 25)
Design and development of Point-In-Time data pulls and dashboard design (Aug 28 - Sept 15)
Website modifications (Sept 18 - 22)
QA and approval of Dashboards (Sept 25 - 29)
Go Live for Point-In-Time Dashboards (Oct 2)
Action Items
- Sarah to set up epics in Jira
5/24/2023 College Accountabilty Phase 2 - Status Meeting Agenda/Notes
Review status of open items from Phase 1 and move all items not completed to Phase 2. https://usfjira.atlassian.net/l/cp/mhLpYtcP
Review Scope, Deliverables and Timeline from Phase 2 Kick Off Meetings to get input or feedback from Valeria Garcia . https://usfjira.atlassian.net/l/cp/bieiqEda Valeria has agreed to scope and deliverables.
Status of analysis for Slalom Consulting engagement Shruti Kumar and Sangita Singh
How far are we on determining metrics and preparing for Slalom; Shruti said we need the schedule before we can progress on completing the data preparation for Slalom. Week of June 5th - 16th, team will prepare.
Questions on metrics definitions, etc.
Update on Slalom Consulting engagement Sarah Thomas
Faculty Productivity Metric (placeholder) Jayakumar Unnikrishnan Shruti Kumar
This is made up of many parts including Citation Impact, Faculty Awards, Percent of publications with international coauthors from outside USF, Percent of publications with national coauthors from outside USF, etc. We don’t have Citation Impact at College level. This is not a metric, but a grouping of metrics.
Internship report - Add slicer by student level
If we can add this as a slicer, it would be very helpful for Career Services. BOG is looking at internships as part of the HIPS. Now this combines UG and GRAD, do you want this separated? Valeria said yes.
Open Discussion
We need to determine a schedule for updating reports/dashboards. Data will be updated at different times during the year.
Valeria said that we do not need to replicate the timelines for the old College Reviews and PPA. We need to think about the user when we make data update decisions. It needs to be as navigable as possible for the user (colleges as well as others).
Sangita said, for example, Research data should be updated monthly. We will need a schedule for stakeholders to let them know when each piece of data was last updated, i.e., student success metrics would be updated at the preliminary benchmark. to document these decisions. Jay suggested a matrix to document and track each metric when it is available for refreshing.
Sangita is working on a matrix already. She also needs to know when this data is refreshed for historical trends and for point-in-time.
Sarah to create an epic to create a schedule. Shruti wants to show and discuss this schedule at the next meeting.
5/11/2023 - Kick Off Meeting
See link to meeting notes on project dashboard.
Project Risks
Resource for web development may not be available when we need him (Chirag). We have mitigated this risk by notifying Chirag and Adam that we will need him for this development. Risk has been averted, Hennadii will be taking on this responsiblity. ✅
Unknown deliverable date for point-in-time reports and college scorecards. The targeted date is October 2, 2023 to deliver phase 2, Point-in-Time reports. This has been changed: Phase 2 will now be the College Scorecards and will be delivered on November 15th. ✅
Project Decisions (date and decision and who decided, most recent on top)
Confirmation of Metrics being used for Scorecards, from 8/11/2023 Meeting
FTIC Freshman Retention Rate: USF, Same College
Academic Progress Rate (2nd Year Retention with GPA above 2.0): USF, Same College
FTIC 4-Year Graduation Rate: USF, Same College
FTIC 4-Year Graduation Rate for Pell
FTIC 6-Year Graduation Rate: USF, Same College
FTIC 6-Year Graduation Rate for Pell
FCS AA Transfer 3-Year Graduation Rates: USF, Same College
Total Research Expenditures
Faculty Awards
Total Research Expenditures per Tenure Track FTE
Citation Impact (upcoming)
Annual Fundraising (upcoming)
8/11/2023 - Instead of doing the point-in-time dashboards, we will be doing the college scorecards first. The colleges are expecting the college scorecards and the point-in-time dashboards are a new feature. The college scorecards should be done by end of October/beginning of November as this is approximately when the first PPA would be delivered.
Metrics as of 6/13/2023 are:
Incoming Student Profile - Average GPA
Incoming Student Profile - Average SAT
Academic Progress Rate
Second Year Retention FTIC
6-Year Graduation Rate for FTIC
6-Year Graduation Rate for Pell
4-Year Graduation Rate for FTIC
4-Year Graduation Rate for Pell
3-Year Graduation Rate for Transfers
Salary of bachelors employed 1 year Post Graduation
Percent Employed post graduation
Percent who are post graduates
Percent of Full-Time Faculty with Terminal Degree (Not a metric for Slalom)
Total R&D Expenditures
Federal Research Expenditures
Faculty Awards
Citations (CNCI) - Normalized Citation Impact
Fundraising
5/31 - Valeria said that not all 5-Year Trend metrics will be done in Phase 2. We are keeping these metrics in line with the metrics Pritish has highlighted for the Strategic Plan. We have reviewed the metrics and the listed metrics will be done for the Point-in-Time and Progress-to-Goal reports/dashboards.
For College Scorecards, phase 3, the list will be the same as for Phase 2 list or possibly fewer metrics.
5/24 - Valeria has approved scope and deliverables set forth in Kick-Off meeting.
Point-In-Time Dashboards for all metrics included in phase 1 - Change, see revised metrics in 5/31 Project Decisions.
Data modeling that will be used for Phase 2 Point-In-Time metrics as well as Phase 3 College Dashboards
Visualization for phase 3 college scorecard to help to determine data model
5/24 - A schedule needs to be set up so that users know when data is refreshed. A matrix was created and approved in June 2023.