Copy
View this email in your browser
We are finally starting to collect early performance data on WIOA outcomes!  While there is still much work to do – both in ensuring more accurate and prompt reporting and in developing the system capabilities to support WIOA performance outcomes – we are excited to see the first information on how YouthBuild grantees are adjusting to these new metrics.  While we hope for more improvement in the placement outcome, we are very excited to see early information on the Median Earnings and Measurable Skill Gains measures.  However, we know this data still isn’t telling the true story, based on the analysis of the MIS Help Desk.  We are hoping this review will trigger programs to take a more studied look at their own outcomes and reporting practices to ensure they are reporting fully.  Please don’t hesitate to reach out to the MIS Help Desk if you think you need more support or training to understand these new and complicated measures.  The data shared below represents a strong start but we know YouthBuild can do better!  ~ Jenn Smith, National YouthBuild Director, U. S. Department of Labor

DOL YouthBuild 2016-2017 WIOA Performance Updates and Timing



DOL YouthBuild and Workforce Innovation and Opportunity Act (WIOA) Reporting

After the launch of the WIOA performance reports in 2018, grantees have been diligently submitting and resubmitting reports and have closed out reporting for Program Years 2016 and 2017!  We now have 2 years of aggregated performance data we can look to for information on how the transition to WIOA is working and how our programs are doing in meeting the goals of WIOA.  This newsletter will review current WIOA grant class outcomes and the timing of performance as a way to understand current WIOA outcomes, identify early challenges, and focus on TA to support stronger outcomes. 


Current Grant Class Reporting Requirements and the Management Information System (MIS) 

It is important to review the landscape of current reporting requirements for DOL YouthBuild grantees because there is confusion in the field. The three currently active grant classes (2015, 2016, and 2017), while all using the DOL MIS online reporting application, have different reporting requirements.  These general requirements are outlined below:

2015 grantees will continue to report using the MIS and view their Workforce Investment Act (WIA) long-term performance indicators in section E of the Quarterly Performance Report (QPR).  This grant class will see no changes to their reporting requirements and will continue to submit the QPR and Narrative through the MIS until their grants end this year.

2016 and 2017 grantees will continue to use the DOL MIS and submit the QPR, but are also reporting on five of the six WIOA performance indicators using the two new WIOA reports: the WIOA Supplemental Report and the WIOA Median Earnings and Measurable Skill Gains (MEMSG) report. These two MS Excel reports are attached to the QPR every quarter. 

*Grantees with both 2015 and 2017 grant cycles will be reporting on both WIA and WIOA performance indicators.  This can pose a challenge for program staff as they enter data and build reports for the WIA indicators for 2015 grant class participants and simultaneously report on the WIOA indicators for the incoming 2017 grant class participants.  It is important to ensure understanding regarding the difference in timing of performance reporting between the two legislative requirements.

 

National WIOA Performance Update

2016 and 2017 DOL YouthBuild grantees have submitted MEMSG and WIOA Supplemental reports for the last four quarters.  After some data corrections and resubmission efforts, the MIS Help Desk, in partnership with DOL, has aggregated WIOA outcomes for Program Years 2016 and 2017.  See updates for each grant class below.
When reviewing this information, there are some important points to keep in mind:
  • Because of the long-term nature of these performance indicators, total grant performance will not be fully realized (especially for second cohorts) until the third year of the grant is finished. 
     
  • Some of this performance will fluctuate as grantees update and resubmit reports.
     
  • These outcomes are for the 2016 grant class, for Program Years 2016 and 2017, but, due to the timing of the indicators, the Placement and Credential Attainment measures were not calculated until Program Year 2017.
     
  • Over 20 grantees with participants in follow-up have submitted 0% placement in Quarter 2 and Quarter 4.
When reviewing this information, there are some important points to keep in mind:
  • Because of the long-term nature of these performance indicators, grant performance for even the first cohorts will not be fully realized until early 2019. 
     
  • As of the end of the first program year of operations (June 30th, 2018), many 2017 grantees did not have any participants that had made it to the time frame of some measures (this is why Placement in Quarter 4 and Credential Attainment are both at 0%).
     
  • Some of this performance will fluctuate as grantees update and resubmit reports.
For both 2016 and 2017 grantees, if you have not reported all placement outcomes properly and have questions about data corrections and resubmitting reports, call or email the MIS Help Desk (866-680-0855 or MISsupport@youthbuild.org).

 
WIOA Performance Calendar and Enrollment and Exit Timeframes for 2016 and 2017 grantees

To better understand our national performance outcomes, we must review the general grant class program calendar and the remaining time grantees have for outcome achievement based on that calendar.  In the following illustrations you will see benchmarks for each grant class mapped to grant months based on national data through November 2018.

DOL YouthBuild Grant Calendar: Based on Forty–Month Period of Performance

Below we see a detail of a DOL YouthBuild grant calendar for a program that enrolls in two cohorts.  This timeline is based on a forty-month calendar which allows for the planning period, the enrollment and exit period, and follow-up period.
Grant Class 2016: Enrollment and Exit Calendar

As of November 2018, the 2016 grant class had enrolled 91 percent of their goal and exited 61 percent of their enrollment goal.  This grant class will have to wrap up training and exit participants soon (within two months) to allow for the required full year of follow-up.
Grant Class 2017: Enrollment and Exit Calendar

As of November 2018, the 2017 grant class had enrolled 48 percent of their goal and exited 19 percent of their enrollment goal.  This grant class has twelve months to continue training and exiting participants to allow for the required full year of follow-up.
WIOA Performance Data Timeline

Another key part of understanding performance is being able to anticipate when performance data will be represented in performance report submissions.  As a general rule, participants are not considered in any given performance indicator until they have reached the timeframe of the metric. These are long-term measures and performance data will be lagging.  All but one performance indicator (Measurable Skill Gains) calculates performance after exit; and for two indicators (Placement in Quarter Four and Credential Attainment), performance is calculated one year after exit.

Below is an illustration of when performance data will be available for participants based on a six-nine month cohort.

*These estimates build in the 4-month planning period; timelines for 2nd cohorts may be harder to estimate due to varying enrollment periods.
 
MIS Help Desk Services

In addition to phone and email support for general data entry questions we offer trainings aimed at helping grantees understand the performance indicators, timing, and reporting processes reviewed in this newsletter. We offer data entry and report trainings weekly!

Data Entry Trainings are held Tuesdays at 2PM ET and Fridays 10AM ET
MIS Report Trainings are held Wednesdays 2PM ET

The MIS Help Desk also offers comprehensive data reviews for individual programs. During these webinars, we will examine your program's reports, review measures, and investigate errors or problem areas. Data reviews could also address any other concerns you have about MIS data entry.

 Please reach out to the MIS Help Desk to schedule a training or review.
CALL 1-866-680-0855OR EMAIL
MISSUPPORT@YOUTHBUILD.ORG

Upcoming Webinar

Please join us for our January webinar on
DOL YouthBuild WIOA Performance Updates and Timing, where you will hear about current national YouthBuild WIOA performance, the WIOA performance calendar, and tools and strategies for reviewing your own performance.
 
Related Resources
 

DOL YouthBuild WIOA Performance Indicators: 2016-2017 Grantees

Building Your Toolbox for Success: Program Design and WIOA Resource Spotlights

Additional Resources
 

Laying a Foundation: Four-Year Results from the National YouthBuild Evaluation is now published.  The DOL YouthBuild National Random Assignment Evaluation Final Report publication can be viewed here.
 
Some highlights of this final report include:
  • YouthBuild increased the receipt of high school equivalency credentials.
     
  • YouthBuild increased enrollment in college, largely during the first two years. Very few young people had earned a degree after four years, and the program had a very small effect on degree receipt.
     
  • YouthBuild increased survey-reported employment rates, wages and earnings, but did not increase employment as measured with employer-provided administrative records, which might not include certain kinds of employment, such as jobs in the gig economy and other types of informal work.
     
  • As with many youth programs, YouthBuild’s benefits through four years do not outweigh its costs.  But it is too early to draw firm conclusions about YouthBuild as an investment, since the benefits accrue over participants’ lifetimes.  Despite its high relative cost, a partial benefit-cost analysis shows that YouthBuild comes closer to breaking even after four years than the two other youth programs used for comparison.
Copyright © 2019 DOL YouthBuild, All rights reserved.


Want to change how you receive these emails?
You can update your preferences or unsubscribe from this list

Email Marketing Powered by Mailchimp