Episode 230 Transcript

The Cult of Pedagogy Podcast, Episode 230

Jennifer Gonzalez, Host

GONZALEZ: This is Jennifer Gonzalez welcoming you to Episode 230 of the Cult of Pedagogy Podcast. In this episode, we’ll learn about a type of surveying that gathers better quality information than the traditional survey. It’s called semantic pulse surveying, and it has the potential to really give your school climate a boost.

GONZALEZ: We have a lot of conversations about what should be done to make schools better. In these conversations, I hear a lot of ideas and I have a lot of ideas, but I think there’s one practice that should be done by any school that wants to improve, and it’s simple and free: Ask the people who are directly affected. The students and teachers in your building are the best sources of data on how things are going in your school; when they feel heard, when they believe that their input matters, the climate of your school just gets better. And one of the simplest and most effective ways to make that happen is with a survey.

I’ve been a big advocate of surveys for a long time. I’ve advised teachers to regularly seek feedback from students and I’ve urged principals to get feedback from their teachers. The kinds of surveys I recommended were pretty much what you’d expect — some multiple-choice questions and a lot of Likert scale items, where you have to choose an item from, say, 1 to 5, or a word on a scale that ranges from always to sometimes to never. I also recommended including some open-ended questions to make room for topics that weren’t covered elsewhere.

But earlier this year I saw a different kind of survey that intrigued me. I was wandering through the Expo hall at the SXSW EDU conference in Austin, Texas, and I walked up to this booth where a few iPads were mounted on stands. On one of them was the question “How are you doing?” and below it were a whole lot of words in circles. 

I couldn’t resist. I started tapping circles, and right away, I was engaged. Words I’d never seen on any survey offered themselves for the choosing. Some were positive: refreshed, significant, interconnected, noticed. Others were more negative: drained, uncertain, filtered, resentful. I tapped a few positive circles because I was having a pretty good day, but I imagined that if I wasn’t, the words in the negative circles would have made me feel seen. I liked that I had the option to choose as many words as I wanted, positive or negative, and the process was fast and instinctive — I never had to pause to think about whether I felt the feeling sometimes, often, or always.

One of the women at the exhibit explained that I was taking a semantic pulse survey, which is built on the idea of the semantic differential, pioneered by psychologist Charles Osgood. You’ve probably seen semantic differential questions before, where two opposite words appear on either end of a scale and you have to choose the spot where your feelings align. For example, a restaurant survey might ask you to rate the service you received on a scale where fast is on one end and slow is on the other, courteous on one end and inconsiderate on the other.

The survey on the iPad applied that same idea, except instead of choosing between just two words, I could pick any words that sounded right to me. If I were part of a large group of people taking that same survey, maybe a group of teachers, all of our input could be aggregated into a report that would show how often each word was chosen, giving a snapshot of which feelings were most prevalent among the faculty that day. Our school leadership could then look at those results and respond: If lots of people chose the word noticed, for example, it would suggest that leadership was doing a good job of recognizing people’s contributions and concerns. On the other hand, if the word uncertain came up often, that would warrant further investigation — it could mean that clearer communication or more support was needed. 

So that explains the “semantic” aspect of this kind of survey. What about the “pulse” part? Widely used in businesses, a pulse survey is one that’s given frequently and can be completed quickly to get a “pulse check” on employees. So a semantic pulse survey combines the word-driven content of a semantic survey with the speed and frequency of a pulse survey.

Surveys like this have the potential to give us much more nuanced information about what’s going on in our schools than other kinds of surveys. They’re easy to create on your own, and today we’re going to look at a few ways to do that. 

Now if you’re in school leadership, you could still do it on your own, or you could use the app I tried at that Expo hall booth. The app is called Alpaca, and the woman I was talking to was Karen Borchert, Alpaca’s CEO. You’ve probably noticed that so far, I haven’t mentioned any sponsors for this episode; that’s because we’re doing something different this time: When I saw Alpaca, I thought, I need to do a podcast episode about this surveying technique, because everyone should be using it, whether or not they use the app. So we decided to devote a whole episode to the topic and have Alpaca sponsor it. Karen is joining me today to talk about how teachers and administrators can create their own semantic pulse surveys, and then we’ll talk a little bit about how Alpaca’s platform helps school leaders better support their teachers. It’s a very cool concept that involves treats, and there’s a special offer for my listeners, so if you’re a school leader, stick around till the end. If you’re impatient and you want to see the offer now, go to alpacapacks.com/pedagogy.

Ok, without further ado, here’s my conversation with Karen Borchert about semantic pulse surveys.

GONZALEZ: We just met a couple, like a month and a half ago or so, at South by Southwest.

BORCHERT: Yes.

GONZALEZ: I was wandering around the expo hall, and there you were with your display. And I thought, this looks really interesting. And so I showed up and we were looking at your product, your platform, which is called Alpaca. Am I saying it right?

BORCHERT: You’re saying it exactly right.

GONZALEZ: Okay. So we’re going to talk a little bit later about what Alpaca does because you do a couple of different things, but one of the things that you do with the app is you provide a place for administrators to give their faculty surveys, to get a pulse on how they’re feeling, you know, and what’s going on so that they can better meet their teachers’ needs. Does that sound like a pretty rough assessment, correct assessment?

BORCHERT: Yeah. That’s, that’s perfect. That’s great.

GONZALEZ: Okay. So the kind of surveys that you guys do are what’s called semantic surveying and pulse surveys. And I had sort of seen things kind of like this before, but I didn’t realize that there was sort of like research behind it and everything like that. So we decided that we were going to do this episode devoted entirely to this, you know, concept of semantic surveys and pulse surveys. What they are, how they work, you know, what the research is behind them and then we’ll talk a little bit about Alpaca at the end and how administrators can do this on their own or they can do it with the help of your app.

BORCHERT: Great.

GONZALEZ: So let’s start by talking about just semantic surveying. What is that? How is it different and what is the research behind why it works?

BORCHERT: Yeah, sure. So semantic surveys are really, it’s just the, it’s right there in the name. They’re based on words instead of a traditional Likert scale or a 1 to 5 scale. So semantic surveys were kind of pioneered by Charles Osgood. He’s a psychologist, and he sort of put forth the research around this in the late ‘70s, in 1979. So they’ve been around for a long time. And the basic concept is that when you ask somebody a survey question like “On a scale of 1 to 5, how do you enjoy this product?” You will get answer, it’s kind of, it’s a little bit difficult. You’ll get an answer. It’s numeric, it’s easy to score, it’s easy to aggregate those numbers. But very often you don’t get much context besides that. So if you instead say, you know, “On a scale of 1 to 5, would you consider our French fries to be very cold or very hot?” You’re going to get some, a little bit more information, right? You’re going to get information not just, you know, if it’s overall level of good, but instead a little bit more about kind of the quality, the perception of product, those sorts of things. So this has been a long, like I said, a really long, long bit of research that Osgood did. And it really basically says we can gain more context, more reasons, and then, honestly, more actionability when we ask survey questions that have real words associated with them rather than only a number. Lots of people combine them. So lots of folks will say, like I said, “on a scale of 1 to 5” but then they have those words, or sometimes people will just have just words that they use to say, you know, “What are the words that you would use to describe — ?” And on the backend of that, they might have some weighting, some sentiment analysis associated with that. But they only, they only put forth the words. So that can be, you know, at the base of it, that’s what it is. It’s a survey based on words instead of numbers, and it is, and the focus of it is to kind of analyze the, you know, what Osgood calls the connotative meaning of emotional attitude towards various things. So when we think about it or like in the context of education and of surveying and of educators, we think about it as that kind of connotative meaning of emotional attitude towards work or towards, you know, sort of any sort of matters at school. So instead of knowing if teachers are, you know, 1 or 5, good or bad, the sort of application of this that we see and that we’ve been kind of working on in our own work is around what are the words that educators use so that it is not, it is not about just, you know, sort of this number, you know. I’m a 2.5. Because it doesn’t feel good to say that, right? It doesn’t feel good to kind of reduce to a number. And so what we see in education because the work is really complex, and you can feel, you know, exhausted and burned out and mission minded and energized all in the same day, the work is better sort of understood with words instead of scales. So that’s, that’s the application that we see of Osgood’s research.

GONZALEZ: I know that there have been, it’s been pretty rare but I have taken surveys like this every once in a while where a company, I think sometimes this will happen, will ask me to choose from a, like a bank, like a word bank, basically.

BORCHERT: Yeah.

GONZALEZ: “Which of these terms describes — ,” and I’ve enjoyed taking it so much more —

BORCHERT: Okay, yeah.

GONZALEZ: — than the feeling of having to choose 1 to 5 because it’s almost like I want to explain a little bit more of how I feel. It’s not that simple.

BORCHERT: Yeah.

GONZALEZ: And being able to choose those, you know. I think some of them I’ve seen have allowed me to choose up to 10 terms. Others have sort of limited a little bit lower. And you see certain words will just pop out at you and resonate much more quickly than, you know. I don’t know. It sort of gives you a vocabulary to describe how something makes you feel.

BORCHERT: It does, and you know, I think in the research we see that a lot of Osgood’s research early on was done with military groups. We see a lot of semantic surveying in a lot of what are kind of considered helping professions, so nursing, teaching, things like that. And part of the reason for it is because to your point of how it feels to take a survey, there’s a little bit more psychological safety in being able to just choose a word that feels like you rather than saying like, “Oh my gosh. What’s going to happen,” or like, “Am I not honoring this profession that I care so deeply about, this mission that I am so, so focused on in nursing, in education and military if I say that I’m a 2?” Right? If I say that I’m a 2, that feels really, like that can be, that can be a really uncomfortable thing to sit with or an uncomfortable thing to say out loud. And so the idea, the semantic survey is in part so that because words give us more context and meaning, but it’s also in part because it feels like you can, you can really say what you mean without feeling like, “Oh, what number does that make me?”

GONZALEZ: Yes.

BORCHERT: And that’s, because that can be, that’s just a really uncomfortable thing to say. And, you know, with educators, you, we find that, like I said, in a given day, you might be a 5 at 9 a.m., and you might be a 1 at 1 p.m., and you might be a 4 again by 4 p.m., right?

GONZALEZ: Mhmm.

BORCHERT: And that’s true of a lot of us in our work, but, you know, I think a lot of the professions like education and nursing that are both physically, mentally, emotionally taxing and are also kind of focused on sort of a mission or, for a lot of people, focus on the mission or kind of a calling, it can feel, it can feel pretty bad to say that. So what you end up getting in the satisfaction survey for a lot of those professions is kind of everything gravitates towards a 4, right? Like, nobody, you know, people are like, “I can’t honestly say that I’m a 5, but I don’t want to say that I’m anything less than a 2. I don’t want to say I’m a 2.” So everybody just kind of comes in somewhere in that 3, 4, you know, range and that’s, you get your outliers but most people just kind of gravitate towards that.

GONZALEZ: Yeah.

BORCHERT: Like a Yelp review.

GONZALEZ: Right.

BORCHERT: They all end up at like 4.5.

GONZALEZ: Yeah. And the thing is there’s so many sort of stories behind why people give different — if I’m, the whole goal of what we’re talking about here is sort of administrators surveying their teachers on how are you feeling about the work?

BORCHERT: Yep.

GONZALEZ: I would imagine that the questions get more fine-grained too, because I’m thinking if I’m a teacher and I’m feeling dissatisfied, even if I choose that word —

BORCHERT: Yeah.

GONZALEZ: — it, the dissatisfaction might be that I have a student who’s really frustrating me or it might be that there’s some new requirement on me that’s being handed down by the administration.

BORCHERT: Yeah.

GONZALEZ: Like, two completely different problems and solutions.

BORCHERT: Yeah, absolutely.

GONZALEZ: I mean, I’m assuming it sort of makes sense to make your questions a little bit more specific then and sort of categorize them so that you can kind of drill down to what is the thing that’s causing this feeling to happen?

BORCHERT: Yeah. I think, I think that either the questions being pretty specific or the answers being pretty specific or descriptive. So to your point, if a teacher’s feeling, you know, amazing because they are, they have an amazing peer team, they’re in lockstep with the other educators in their building. They feel supported and cared for, that’s awesome, right? If they feel, but if they feel great, if they’re a 5 because they feel like their leadership has provided this super healthy environment, this environment that is focused on physical, mental, emotional health, those are two different kinds of good, right? And you kind of celebrate them in two different ways, right? You celebrate this team of teachers that is taking good care of each other or you celebrate the fact that your team is, is focused on their, you know, mental and emotional health so that you could celebrate those two things. And on the other side, you can, you know, with really descriptive words, somebody who’s a 1, right, if they are a 1 because they’re frustrated, because like you said, because of some new requirement or even because of, you know, something’s going on with a student or a family, frustrated is a super different emotion about work than lonely. But they might both register as a 1, right?

GONZALEZ: Yeah.

BORCHERT: And so, and lonely you mitigate in a totally different way than frustrated.

GONZALEZ: Exactly.

BORCHERT: You know, if you have kids, I have kids, if my kids are lonely versus frustrated versus, you know, sad, versus angry, those are just all super different emotions that we handle differently, and I think that’s the same at work for, honestly, I think so for all employees, but I think particularly in education that’s the case.

GONZALEZ: Yeah. So when, if somebody’s putting together a survey like this, a semantic survey, is it better practice to put more words there just to, like I’m assuming choosing the words is probably the whole skill of making these kinds of surveys, is picking the right words.

BORCHERT: Yeah, yeah. So there’s, there’s certainly that. The good news about semantic surveys is, you know, especially if you are doing them on really regular cadence is you can, you know, you can change those words that are in there and kind of hold them into some buckets. So, you know, if you say, okay, well this, these buckets are, this bucket is about sort of how we feel about leadership and this bucket is about how we feel about our team and our peers, and this bucket, you know. If you bucket them, then if the words that are in there are not resonating or not getting used ever or are just, or just don’t feel like when you look back at the results of that survey and you’re like, huh. That’s interesting. I don’t, I don’t think that my team saw the word “frazzled” the same way I see the word “frazzled” or, you know, those sorts of things. So I think having a rubric, having clear definitions for the words that you’re using and how you’re using them, not to present all that information in this survey just for your own information is good. And then the other way to fill that bank of words is to start by asking, asking open-ended questions. So, and there’s some really good ways to do that. So asking open-ended questions, asking an open-ended question like “Everybody get out Post-it notes and everybody put one word per Post-it note. Use one word you would use to describe work in the last week and, and make five Post-it notes.” And put all the Post-it notes on the wall. You’ve basically made an educator informed semantic survey word bank just with that.

GONZALEZ: Right.

BORCHERT: And, and that gives you a little bit of information. That can kind of change over time. So sometimes combining that with, you know, some open-ended work can be good. This is a really excellent use of ChatGPT is to do a survey that says, you know, “Tell me everything about how you feel about work right now,” open ended. Just tell me anything you want to tell me. Totally anonymous, all of that. You could do that in a single word or a single question survey. Pull all of the data from that and ask ChatGPT to create a list of a hundred words based on, based on your teachers’ sentiment.

GONZALEZ: Yeah.

BORCHERT: And based on this input, and see what it comes up with and see if it resonates, is an interesting OpenAI use too.

GONZALEZ: Mhmm.

BORCHERT: But that’s, there’s some really good ways to do that.

GONZALEZ: Okay. So, and I think we sort of, I skipped over this, and I’ll probably say this when I write up an introduction for this interview, but I feel super strongly that anybody who serves people in any way, whether you’re the leader of a team at a school or you’re a teacher of students or whatever that you should be surveying your people on a pretty regular basis because that’s sort of the best feedback that you can get. And so you’re talking about maybe starting with like an open-ended one where you can just gather all kinds of information and then maybe turn that into a semantic survey where you’re [inaudible] down to specific words to get an idea of how many people feel this way, and what’s the general temperature of the building?

BORCHERT: Yeah.

GONZALEZ: Yeah.

BORCHERT: Yeah, and I’ve even heard of some folks who will send out via text a one-word or like a one-question, you know, sort of quick survey to say, please record a voice memo, just record this with your voice. Talk to me for anywhere from 30 seconds to 5 minutes about how you’re feeling at work and what’s going on in your world right now. And it will transcribe it, right, into a text message, and you can pull all of those into [inaudible] so I’ve heard that before. That’s an interesting way to do it because then really what you’re doing is just, surveying is just listening, right?

GONZALEZ: Yeah.

BORCHERT: And any profession who has people managing other people, listening to the people you’re managing is absolutely paramount to building high trust and a great relationship together. And so surveying is one way of listening. Listening is another way of listening, right? Just having a conversation, asking somebody to tell you, you know, talking to them in a one-on-one meeting or in a, you know, kind of a check-in. All of that is, all of that is pieces of data that then you can use if you want to create a little bit more of a, you know, sort of semantic survey with a word bank like we were talking about.

GONZALEZ: Yeah.

BORCHERT: Yeah.

GONZALEZ: So we’ve sort of covered the idea of semantic, but we also keep kind of referring to this idea of doing it kind of regularly.

BORCHERT: Yeah.

GONZALEZ: So let’s talk about pulse surveying.

BORCHERT: Okay, great.

GONZALEZ: What is a pulse survey?

BORCHERT: Okay. So a pulse survey is, kind of exactly, also what it sounds like, which is to say it’s a survey that is done to take the pulse of an organization, a group of people, employees, customers, on a really regular basis. And so the idea here is, you know, ask, ask quickly and ask often. And it’s a, it’s a simple way to do a survey. In education, we’ve all probably seen or done or heard of a climate survey or a sort of culture survey, some of those things that are often done once or twice a year. They take a, you know, they take a good 20 to 40 minutes sometimes. They’re really helpful. They’re a great set of information and data for a school and a district, and they’re kind of all done at the same time so you get, you know, kind of that control in there as well. But, they are a little bit, it’s a little bit more difficult to quickly action on things that might not be going well or on things or on great things that might, you know, might be worthy of celebration —

GONZALEZ: Yeah.

BORCHERT: — if you get that information back just once a year. So with the pulse survey, the whole point of a pulse survey is giving small bits of information that you can act on quickly, and then you can take that pulse survey again and understand, you know, okay did that little thing get better? Or did that little, how is that little thing going? It’s a really good way to kind of sense that, sort of the idea of, sort of the curve around sort of teacher well-being and teacher morale that happens throughout a school year. A pulse survey lets you kind of really check in with educators at each of those moments. So in the kind of traditional sort of slump months and maybe October or March or those kinds of times where it kind of feels like, you know, you’re a long way from the next big break.

GONZALEZ: Yeah.

BORCHERT: That can be a really great moment to say, “Hey, what’s one thing that would help right now?” or “What’s one way you’d like to celebrate this week?” or “Who’s somebody who’s helped you?” Any little questions, pulse surveys can be done with these same exact questions every time, but they can also be done with a little bit of a rotation of question where maybe you have a couple of questions that are, you know, asking the same thing each month or you might get ones that are, and then you add in, you know, one or two little questions here and there that would give you a little bit of a wild card, like, you know, how are you feeling about our upcoming testing season? Or what’s something that we could do since, what’s one thing you’d like to do with students when the weather’s so cold? Right? You could do some of those kind of little wild card questions as well that can kind of keep that.

GONZALEZ: Yeah.

BORCHERT: Pulse surveys tend to have really good participation. It’s one of the big reasons people do them because they’re fast and they’re easy and often, if they’re done well, the thing that is asked about is actioned upon in a short time frame where you can see it, and that’s really the main benefit of a pulse survey.

GONZALEZ: Right.

BORCHERT: It’s that the person who’s taking that survey, the No. 1 reason people take or don’t take surveys is not their frequency and it is not their, the questions, actually. The No. 1 reason people either choose to continue taking a survey or don’t choose to continue taking a survey is whether they feel like their answers were heard and acted upon.

GONZALEZ: Yeah.

BORCHERT: And that’s, that’s a whole ball game. So pulse surveys, the whole point of a pulse survey is just to give a little bit more constant feedback loop that creates opportunities for action and opportunities for communications and conversation.

GONZALEZ: Yeah, yeah.

BORCHERT: Yeah.

GONZALEZ: So we’re going to be talking, because your app actually facilitates a whole lot of this. But if somebody’s just listening to this idea, and they say, I don’t want to mess around with an app. I just like this idea of doing these surveys.

BORCHERT: Yeah.

GONZALEZ: What is a way that a school administrator could do this just on their own?

BORCHERT: Yeah. We were actually just talking to an administrator last week. He was calling us about our software, but he, he said, “I read about what you did, and I wanted to do it immediately so that day at my staff meeting,” he said, “I got out pieces of paper and I said, ‘What are five words you would use to describe our school culture today?’” And they all made Post-it notes or little slips of paper with all of them, and then he said, “Okay, pull out one more Post-it note and answer this question for me: Is there anything that you would like me to know about you as a person right now?” And he said, “We did the whole thing anonymous, but we did this really, really quickly, and that idea,” and he said, “We just did it analog. I didn’t, you know, we didn’t do anything on our phones or anything else.” So he said, “I had this whole wall of words that people used to describe our school culture, and I knew,” he was like, “I looked at it and I knew which ones I liked and which ones I really didn’t like.” And then he said, “But I was most surprised by the Post-it notes about one thing they wanted me to know. And I told them they could sign it or not sign it, and that would be okay, and so they did it that way.” He said, “I understood so much more about what my staff was going through personally in their lives, and they were really willing to open up.” He’s like, “Somebody filled three Post-it notes front and back.”

GONZALEZ: Wow.

BORCHERT: I was like, oh, they had a lot to say.

GONZALEZ: And all it took was asking.

BORCHERT: All it took was asking. And so that’s a really easy way to do it. Post-it notes, really simple like quick ways of asking just one question at the beginning of a staff meeting. You can do it as an ice breaker. Those, those are really good ways. We have a, we have a resource that is really widely used that is kind of a version of this that’s called start stop continue. It’s a really, really commonly used management practice in kind of, in the private sector, which is sort of where I come from. But it’s, we do it with color cards, so green, yellow, and red, and we say, what is, you know, we give everybody out three cards: One green, one yellow, one red. And we say, you know, what’s, on the green, what’s one thing you want to start doing? And then on the red, what’s one thing you want us as a school to stop doing? And then on the yellow, what’s one thing you want to continue doing? And we’ve seen this in place in tons of schools where even an administrator will put an envelope on their door, and they’ll just hand out the cards at a staff meeting, and they all say start, stop, or continue. And they’re completely anonymous, and they say, just fill out each of your cards, and then put them in the envelope whenever you want to in my office, and then at their next staff meeting, they picked, the administrator picked one of each card that they were like, “Okay, I heard you. I read them all. I heard them all. Here is one thing I’m going to start, one thing I’m going to stop, and one thing I’m going to continue based on your feedback.”

GONZALEZ: Yeah.

BORCHERT: And that is, that sounds really simple but that’s, that takes less than five minutes to write those three cards.

GONZALEZ: Right.

BORCHERT: It gives, and then it honestly gives a principal a roadmap for all kinds of ideas that they could start. It gives teachers a seat at the decision-making table on things to start, stop, or continue, and it’s an easy way for educators to feel really heard in the building. So those are a couple ways we’ve seen.

GONZALEZ: Yeah. And then if we wanted to, like, go higher tech, you know.

BORCHERT: Yeah.

GONZALEZ: What would be some ways of doing that?

BORCHERT: Yeah, cool. So we, there’s a couple of different ideas. So you can use a really basic survey platform like SurveyMonkey or Typeform. I really like Typeform. And send your teachers, you could send your regular teachers every day. We’ve also seen this work really well with sending a quick little survey to substitute or guest teachers after they work at your school with just a three-question survey.

GONZALEZ: Yeah.

BORCHERT: With substitutes, here’s a three-question survey about your day-to-day at our school or with your teachers, here’s a, you know, it’s kind of the Friday, it’s kind of the Friday check-in, and here’s three questions about, about your week this week and how things, how things have gone for you this week. You can even use that start, stop, continue in something like a SurveyMonkey or a Typeform. And then as I mentioned, when you use survey things that are online like that, you can do some really interesting things with it to analyze your results, including some of those survey platforms actually have a little bit of analysis that’s kind of baked right in and you can look at that. You can also dump all of your results into a spreadsheet or a document. And like I said, you can take that over to ChatGPT and say, “I surveyed all of my teachers with these three questions. Here are their answers. Please provide some analysis and some ideas for how we might support our teachers best this week.” And it is amazing what, you know, it’s not perfect, of course, but it does give you, like, kind of an aggregation of that data, and that can be a really, really, really simple thing to do.

GONZALEZ: Yeah. I think it’s sort of like, it’s sort of a given that anybody listening to this, I know a lot of teachers are listening and thinking, well, I’m not an administrator, but these kinds of surveys would also work great as a teacher working with students.

BORCHERT: Yes.

GONZALEZ: So how could that look in a classroom?

BORCHERT: Yeah. Well, there’s a couple of ideas overall. One that I’ve seen is kind of a question, sort of question of the day on the whiteboard or whatever where the only, the answers are just, are just simple words. So you could say, you know, how did you feel about this morning’s lesson? Or what was the best thing about recess today? Or what’s your favorite lunch at school? And all of those things, and then kids can come up with a dry erase marker and just, you know, write their answer, and it kind of creates a word cloud or a collage, right there on your white board. You could even have like a little section for it on your white board that just sort of stays. Kids don’t have to answer it. Kids can answer with one word or five words. They, you know, it’s completely optional, but over time, what happens then is kids see that, it’s like oh, I’m not the only one that thinks that, like, the best thing about recess is, is, you know, running on the track. I thought I might be the only one. Or, you know, my favorite, my favorite lunch is chicken nuggets too. And so there, it kind of just creates some commonality. And so they can be really, really kind of fun, safe, easy questions. They don’t have to be like, how are you feeling at home? Like, you know. Some of those words might, those might not be as easy to do in that kind of setting, but just creating trust and using, you know, sort of using our words to express how we feel. You know, “What’s one emotion you would use today to describe yourself?” is one. There’s some, there’s some cool ways to do that. Obviously, like, things like, in the classroom, things like zones of regulation, kinds of exercises where I’ve seen that with popsicle sticks or whatever, that it’s red, yellow, blue, green, and each one has kind of a different, like, level of sentiment or emotion, and when kids come into the classroom, they put their popsicle stick in the right color bucket, and that gives, that is, that is basically a pulse survey done with, done with students on a regular basis. So there’s some, there’s some really, really cool things that teachers already do. I actually think a lot of things that teachers already do are great things to adopt for administrators.

GONZALEZ: Yeah, yes.

BORCHERT: There’s some really, really cool things that teachers are doing. There’s like the word wheels where there’s, you know, kind of all, kinds of words that a kid might use, and you can kind of say, you know, what are the, if you’re meeting with a student one-on-one, like how are you doing? Like, “What words in here would you use to describe how you’re doing today?” It’s a really nice way to just give kids words to use.

GONZALEZ: Yeah. So let’s talk now about Alpaca.

BORCHERT: Okay.

GONZALEZ: What does your platform, how does it work? And how does it make surveying easier for administrators? And let’s clarify that currently it is designed only to be used by administrators with their staff, not for teachers to use with students.

BORCHERT: That’s right.

GONZALEZ: We had talked about this earlier, and you said that you could see it being used that way, but right now it’s not set up for that.

BORCHERT: Right, right.

GONZALEZ: So, tell me about Alpaca.

BORCHERT: Yeah. So the, I think the basic thing about Alpaca is we liked the concept of semantic surveys and the concept of pulse surveys so much that we decided to combine them. So we, so what we started with overall was, you know, kind of a, like I said, an open-ended survey with a lot of educators that were engaged with us through, we actually started with an offering that provided kind of a subscription care package to teachers on a, on a monthly basis. And with that care package, with all of our initial schools and teachers, one of the things that we always, we always included a little, a little kind of satisfaction survey in that, in that. And we would say, you know, what are the kinds of things you need more of or less of at school and, you know, what did you think? And then we asked this question, how are you doing? We just asked that question, how are you doing? And we got a huge variety of, of responses, and a ton of words. And over a couple of years, we actually gathered about 16,000 inputs from educators about how they’re doing, about things they need, about their favorite stuff, about how they like to communicate. They don’t like to be called. About how they, about how they, the ways they like to celebrate, things they like to do together. We just, we just pulled a huge body of sort of research and input from, that were actually straight out of teachers’ mouths as far as like, you know, just the words teachers use. So one of our core values as a company is “listen to teachers.” So that’s where we started, and then we took that, that kind of dataset, you know, sort of fully anonymized all of those things and we built, we built basically what we call a semantic pulse survey. It’s a survey that goes out once a month to everyone in the building. Some of our schools will do it as part of their team meeting or their staff meeting. Some of them will do it individually. It’s a mobile-friendly survey, so it’s, again, the whole point is you’ve got to be able to do it in less than a minute, really fast, really easy. We try to make it pretty fun. It feels, it feels like a lot of fun, and it asks the question, sort of the main question in this survey, there are a few of them, there’s like four or five questions, but the main one is, “How are you doing?” And we present a bank of words that are randomized, for every time you use it, you get a random bank of words. And you can pick as many as you want. So when you pick a word, it actually kind of floats down to your own little word bank, and then it’s replaced with other words up top. So you can also refresh and get all new words and see a completely new, new set of things. And you can pick as many as you want. And then we also offer an open-ended kind of answer at the end that says, you know, “Is there anything else you want to share about work right now?” And that’s, you know, that’s completely open-ended. They can say whatever they want. We associate the survey with the building, but not with any individual. You don’t put in an email address or a name when you take this survey. That helps keep it something that’s just simple to do. And then we aggregate that data. We have, we have built a model that creates a level of sentiment around each word so how positive or how negative a word might feel. And then we’ve also associated those words with specific, kind of specific drivers of educator well-being through the research at the University of Chicago Consortium on educator and leader well-being. So we’ve done that, and we’ve kind of put all of that together, and so each month, after you get this survey, your principal will basically get a little data report, a very brief data report, and it, and we don’t really aggregate things into scores or indexes. Like I said, nobody wants to feel like they’re, you know, a negative 13 or a 2.5 or anything like that. So we instead just say, hey, here are the words that your teachers are using to describe work right now. Here are the top five really, really positive words and maybe the top five more challenging words. We say, here’s one thing in your, in your building you might think about celebrating right now, and one thing that may be a good place of focus. And then we do ask that question, what’s one thing that would help the most right now? The No. 1 answer is always “more time for planning” or very often. But we always say, you know, sometimes it’s actually really interesting to see what the No. 2 answer is, which is sometimes “time with peers” or, you know, “better connection to my students.” And so that kind of gives, again, another little actionable piece of information. And then we, but we just give you the, the real words that teachers are using. And one of the reasons we do that instead of trying to roll up a bunch of, you know, indexes or scores to say, you know, here’s where your teachers are in their overall level of well-being, here’s their score. You know, people will, will kind of feel like, oh, I want that score, you know, I want to know what the number is. But at the same time, you actually get, again, so much more, so much more information from the words instead. And not to get fixated on that score, we kind of tell people all the time that scores can be a wall, but words can open a window, like, they can open, open a door for a conversation. And so one of the things that we say is, you know, here’s some questions that you might ask your team. Like, you know, we have a few different, a few different words this month that indicate a level of kind of isolation or loneliness. What is everybody’s idea for how we might help make sure that nobody feels alone here, right? Or, you know, and sometimes people will see stuff in their data that we would never have known because we don’t know what’s going on at their schools.

GONZALEZ: Right.

BORCHERT: So we work with the district. Right now who did, you know, they do this survey on month three, elementary schools every month. And one month, all of the words were, or a lot of the positions words were, like, I felt clued in, informed, engaged, involved. And we, we were like, oh, here are the words. And their head of, their head, their supervisor of principals said, “I am so happy to see those words.” And I said, “Oh, tell me more.” And he said, “Well, we’ve had a districtwide training and focus over the last couple of months around creating opportunities for shared decision-making among teachers and helping principals help their teachers feel like they have a greater seat at the table and that they’re part of the decision in their building.”

GONZALEZ: Yeah.

BORCHERT: And so he was like, “To see teachers saying that they feel clued in and engaged and involved is really,” he’s like, “that’s really, really great to see.” We would never have known that, obviously —

GONZALEZ: Right.

BORCHERT: — from looking at the analysis. So our goal always is to just turn back the information and the data to you, and then to be able to give that information and show it over time.

GONZALEZ: Right. You guys originally started with these care packages. Can an administrator just do the surveying piece without doing all of, or is that just together?

BORCHERT: Yeah, no, we do it both. We, we have, we still have what we call our celebration pack, so their little reward and recognition packs. We have Alpaca Pulse, which is our pulse survey kind of product and system. And you can purchase one of those or the other, or you can purchase them together as a little system.

GONZALEZ: Okay, okay.

BORCHERT: So what a lot of districts are most, kind of most popular thing that districts or schools do is they will do pulse. We price it on the basis of how many certified teachers you have in the building, but then you can use it for everyone in the building, your paras, your assistants, your school nurse, everyone can take this survey and we give you that information back, however you want to do that. And then what they’ll often do is purchase a certain number of packs per month. Every pack, like the packs each month are themed, and they’re different every month, and they follow along with some concept or idea around educator well being. They’re kind of like, it’s kind of like Stitch Fix but for teachers.

GONZALEZ: Yeah, yeah.

BORCHERT: So they include some supplies that teachers are, you know, often run out of or feel really great to have a replenishment of. They always involve snacks and treats. And then they often involve a little bit of, you know, sort of something kind of, something kind of special, you know, a really cool digital timer or, you know, something that is like, oh cool. Kind of a new product for your classroom, classroom organization, classroom decor, stuff like that. And so we always, the packs always have that. They’re different every month, and most of our schools, like I said, they’ll do the survey, and then they’ll get, say, 5 packs or 10 packs a month, depending on the size of their school. And then they’ll use those together. So they’ll do their staff meeting. They’ll say, okay, quick, everybody get out their phones. We’re going to do pulse. Everybody does pulse, and they say, okay, we have five packs to give away this month. This month we’re giving them out to teachers who were nominated by other teachers in the building just because they, we’re giving these away to five teachers who help, who really, really help each other. Maybe, you know, five who really helped the new teachers, or things like that. Or some months I’ll say, like, we’re going to do five for kind of unsung heroes in our building, people who may not get the recognition that, you know, or be showered with student gifts at Christmastime or things like that. So, you know, five unsung heroes and this is what we’re going to do. So we, we do put that together as kind of a package, and it’s, kind of as a package deal, our system, because our idea is that listening to teachers should be easy and recognizing teachers should be easy, and that should be something that, and it should be consistent, something you do all year-round in order for it to really feel really great. So we kind of put it together. We think it works best as that, but we have tons of folks who only do the packs, and we have tons of folks who only do the surveys, so.

GONZALEZ: Got it. Okay, that’s good to know.

BORCHERT: Yeah, yeah.

GONZALEZ: I didn’t know if they were, they were tied together or not. 

GONZALEZ: Hey, I’m gonna cut in real quick to let you know that Alpaca is offering listeners of this podcast 15% off of a year of Alpaca’s Pulse Surveys. To grab this offer, visit alpacapacks.com/pedagogy.

GONZALEZ: So if people want to learn about what you do at Alpaca, if they want to find out more about the surveying or the packs, where do they go?

BORCHERT: Great. Well you can follow us on any of the socials. We’re generally at alpacapacks on Facebook, Instagram, Twitter, etc. But the easiest way to find us, a little video about how it all works, all of those things, is at our website, and that’s just www.alpacapacks.com. And that’s, that’s the easiest way to get to us. And then I’m on LinkedIn as Karen Borchert, and I would love to talk all day with you about semantic pulse surveys and why I think they’re great and what, how you might use them. So I, you can find me on LinkedIn anytime or just at karen@alpacapacks.com and would love to, we would love to talk.

GONZALEZ: Awesome. All right. Karen, thank you so much.

BORCHERT: Yeah, thank you so much.

To learn more about semantic pulse surveys, visit cultofpedagogy.com, click Podcast, and choose episode 230. To get a bimonthly email from me about my newest blog posts, podcast episodes, courses and products, sign up for my mailing list at cultofpedagogy.com/subscribe. Thanks so much for listening, and have a great day.