CalPEPAL Brief Report - Oct 2022

Background

In addition to simplifying California public school funding calculations[1], the Local Control Funding Formula (LCFF) departed from a state-mandated accountability system wherein legislators dictated public schools’ expected goals and progress. Premised on Governor Brown’s adherence to the principle of subsidiarity (Bae and Stosich 2018; Wright 2017), the 2013 policy designates local educational agencies (LEAs), i.e., school districts[2], as the primary budgetary decision-making units that must strategically plan future expenditures before receiving annual funds. With input from community stakeholders, school districts must write and publish a Local Control and Accountability Plan (LCAP) each year. In their LCAPs, district administrators identify goals, actions, services, and predicted expenditures aligned with statewide education priorities to improve outcomes for all students, and especially higher-need groups like low-income students, English learners, and foster youth.

Although the State Board of Education (SBE) has revised the LCAP template multiple times[3], the documents have always required districts to explain how they will increase or improve services for higher-need student populations in what we refer to as Section 3[4].In this section, districts are asked to describe (1) how actions for the entire school will affect the needs of foster youth, English learners, and low-income students and how these actions are effective in meeting the goals of these students, and (2) how these services and supports are being increased or improved by the percentage of improvement reported by the school district. Figures in this report’s Appendix display examples of one school district’s Section 3 for School Years 2016-17 and 2017-18, and reflect template formats the SBE adopted for early and more recent implementations.

In this CalPEPAL Brief Report, we use quantitative text analytic techniques to examine a large sample of California elementary and unified school districts’ Section 3 responses from 2014-15 to 2021-22, describing over-time changes in how districts planned to allocate their supplemental and concentration grant funds.

A coarse way to examine the attention school districts paid to unduplicated pupils is to look at the total number of words and unique terms used in Section 3. As illustrated by Table 1 and Figure 1, on average, school districts used increasing numbers of words–called “tokens” in the text analysis literature[5]–in these descriptions each year. Notably, changes to the LCAP template do not appear to account for this growth despite significant overhauls to the plan’s format throughout implementation. The LCAP template was substantially altered between 2016-17 and 2017-18, yet the mean number of tokens used in Section 3 grew only by approximately four words.

One exception to the trend of increasing description length came in the 2020-21 school year, when the state legislature temporarily replaced the LCAP with the Learning Continuity and Attendance Plan (colloquially referred to as the LCP) for school districts to plan their shifts between distance, hybrid, and in-person learning triggered by the COVID-19 pandemic[6]. On average districts included fewer tokens in their LCPs’ Section 3, potentially because they deprioritized writing in-depth administrative plans amidst other pressing day-to-day obligations during this turbulent period. The most dramatic change came in 2021-22, when average Section 3 descriptions exploded in length. The average school district included approximately 1,250 tokens in their 2021-22 Section 3, nearly five times the average number of words written during LCFF’s inaugural year.

Figure 1. Average Number of Total Words and Types Over Time

This shift in length is further evidenced by the distributions of tokens displayed in Figure 2. In 2014-15, this distribution was highly skewed, indicating that a large majority of districts wrote very few words about how they planned to support English Learners, students from low-income families, and foster youth. Alameda Unified School District’s LCAP included the longest Section 3 description in 2014-15, with just over 1,900 tokens. Over time, the distribution of token counts has gradually shifted as more districts began writing longer descriptions. By 2021-22, over 20% of districts wrote more than 2,000 tokens in their Section 3 descriptions.

Table 1: Descriptive Statistics of Length, Uniqueness, and Complexity by Academic Year

Alongside lengthening descriptions, districts also expanded their vocabulary every year apart from 2020-21. The average district used approximately 117 unique terms, referred to as “types” in the text analysis literature, in its 2014-15 Section 3 description. By 2018-19, the average number of types grew to approximately 210 words, and by the 2021-22 academic year this metric increased to 322 (see also Table 1 and Figure 1).

Figure 2. Distribution of Total Number of Words by School Year

Longer LCAPs have not been regarded positively by all, as some stakeholders have argued that LCAPs’ wordiness and use of jargon create a barrier to accessibility among non-experts like parents and community members. One metric to gauge a text’s complexity is the type-token ratio, which divides the number of unique tokens (types) by the total number of tokens contained in the document (Richards 1987). Despite becoming longer and containing more unique words, districts’ Section 3 descriptions slightly decreased in complexity, on average, between 2014-15 and 2020-21 before dramatically dropping in 2021-22. Figure 1 helps explain this trend: growth in the average total number of words outpaced growth in types, particularly between 2020-21 and 2021-22. This suggests that, even though districts wrote longer responses and used more new terms in Section 3 descriptions over time, they have also been repeating these terms and phrases to an increasing degree.

Trends in Content 

Summary statistics helpfully tell us about the structure of texts, but they cannot tell us about what authors write in them. Examining the most common words can shed some light on the latter, however. Figure 3 displays, in declining frequency, the tokens that account for at least 0.05% of the words included in that year’s LCAPs[7]. Although the number of tokens and types have grown significantly over LCFF implementation, the top words per year have remained relatively consistent with a few small differences in order. For example, in every year “student” and “school” are the two most common words, accounting for a combined 6% of all words used across each year’s LCAPs. Other tokens associated with education, like “teacher,” “achiev-,” “academ-,” and “educ-” itself also regularly appear among all years’ most frequent words . Taken together, these terms’ prevalence suggest that, despite LCFF’s whole-child approach to education, districts are writing their Section 3 descriptions with a focus on allocating funds to raise academic achievement.

Figure 3. 25 Most Frequent Annual Words in LCAP Section 3

Drawing on our substantive knowledge of the LCAPs, we also identified expected groups of related terms. For example, words associated with higher-need student groups such as “undupl-” (the root of “unduplicated”); “low” and “incom-”; “foster” and “youth”; and “English” and “learner” are staples in Section 3. Similarly, words associated with the funding formula like “supplement,” “concentr-,” and “fund” can be found among these terms. Districts also frequently use directives about how they expect to use their funds to support these higher-need students like “provide,” “increas-,” “support,” “service,” and “action.” These groups of words can also be found in the template’s instructions, which suggests a tendency among district personnel to reuse LCAP template language in their responses.

Some notable exceptions buck these trends, however. For example, we observe frequent use of technical language associated with LCFF (e.g., “proport-” and “percentag-”) only during LCFF’s inaugural year. The shift away from frequent use of technical terms by 2015-16 is likely explained by districts’ increased efforts to use accessible language in LCAPs following LCFF’s initially confusing and quick timeline for implementation (Arseo 2021). As we explain elsewhere[8], CDE devised the minimum proportionality percentage (MPP) for school districts to measure their annual expected increase in services and supports for unduplicated pupils. Whereas education administrators often approached the LCAP as a technical strategic planning and/or budget alignment tool, community advocates stressed the importance of its readability for popular stakeholders like students’ parents and families to engage in decision-making processes (see Knudson et al. 2017 for a discussion of these debates). Because the LCAP planning process is intended to promote “authentic stakeholder engagement,” state assistance providers quickly recommended districts minimize their use of technical language like “MPP'' in their plans (Knudson 2014). On the other hand, both “distanc-” and “access” rank among the most frequent terms in 2020-21’s LCPs. Their frequency reflects districts' focus on the unique conditions facing higher-need student groups during this period, chief among which was access to technology. Perhaps surprisingly, given the large-scale return to in-person schooling in 2021-22, “access” remained a frequently used term in these most recent LCAPs, perhaps indicating that concerns about access have continued in the wake of California schools’ shift to remote instruction.

One unexpected term elicits further reflection, however. In every year, California school districts frequently referenced “parent-” (although the term is replaced by “famili-” in the 2020-21 LCP as standard terminology briefly shifted before reverting back to “parent” in 2021-22). Although a deep investigation is beyond the purview of this brief report, this term’s prevalence likely reflects the CDE’s explicit incorporation of Karen Mapp’s dual-capacity building framework into the state’s pedagogical approach (California Department of Education 2017). Mapp’s framework emphasizes the importance of building relationships with students’ families to better support them, and argues that by building families’ capacities as their children’s advocates, especially those from historically disadvantaged communities, educators can gain critical support to ensure these students’ success in classrooms and at homes.

Conclusion

In sum, our research points to some important structural changes in LCAP Section 3 descriptions. Over time, school districts have gradually written longer descriptions with more unique terms. However, the complexity of these descriptions as measured by average type/token ratios has declined, with a particularly steep drop-off between 2020-21 and 2021-22.

Longer descriptions that use more unique words do not necessarily connote “better” expenditure plans. On the contrary, long descriptions featuring repetition of key phrases may suggest that school district administrators are approaching LCFF’s accountability mandate as a matter of bureaucratic compliance rather than tailoring their plans to their student populations’ unique needs or proposing innovative approaches to increasing equity. Moreover, longer descriptions may be unnecessarily arduous for community stakeholders to digest, thereby limiting the LCAP’s potential as an effective tool for public communication. Other researchers and advocates have emphasized LCAPs’ lack of transparency, inaccessibility, and transformation into rote compliance documents (Fullan 2015; Hahnel and Humphrey 2021).

Despite substantial shifts in the structure of the Section 3 descriptions, their content is largely what we might expect from responses to the question posed in the plan’s template: how do school district administrators plan to spend funds earmarked to improve and increase services for higher-need student groups? Aside from notable exceptions during LCFF’s inaugural year and in the LCPs used to plan for distance learning, the most frequently used words remained consistent throughout the first eight years of LCFF implementation. This trend suggests districts largely align their plans with the templates’ directives. However, frequent references to the students’ primary support systems like their parents and families suggest a general shift from prior eras of teacher-focused state accountability to LCFF’s more encompassing approach to educating youth. In future CalPEPAL Brief Reports we will extend this broad, descriptive overview by investigating the content of districts’ Section 3 descriptions in deeper and more nuanced ways. For example, we will examine variation in district plans by local demographic and economic contexts, political landscapes, prior trends in schooling outcomes, and other relevant school district features.

Appendix: Example LCAP Section 3 Descriptions

Figure 1A: Excerpt from Davis Joint Unified School District’s Local Control and Accountability Plan, 2016-17.

Figure 1B: Excerpt from Davis Joint Unified School District’s Local Control and Accountability Plan, 2017-18.