The Cost of Field Testing

http://student.collegeboard.org/?student

     This week our Freshmen students took the first two tests in the latest version of Texas' standardized testing program STAAR (State of Texas Assessment of Academic Readiness).  At the same time our Juniors were being tested in a field test on the same subject areas.  I had the opportunity to be one of the proctors for the Junior tests.

     I want to make sure that I say, up front, that the staff here tried to take these field tests very seriously.  We tried to get the students "excited" (if that's even possible) about taking them.  We went through all of the steps you do when you are administering the test as it is required by law.  The one thing we could not control was how the students would act during the testing.

     Student conduct on field testing drives me crazy and makes me do things like write this post.  As someone with a degree in mathematics (specifically statistics), I question the validity of data obtained from this "random sample" of students taking the tests.  Yes, you can randomly select schools to do the field testing.  But the data will be incredibly skewed by the attitudes of the test takers at those schools.  And, high schoolers WILL skew data if they decide to do that.

    Anyone who has seen Freshmen take a standardized test knows that there will be a certain percentage of students who will "christmas tree" the test.  Christmas treeing is quite fascinating.  Some students will select a letter to bubble in, say an A, and the entire test will be that letter.  Most students just bubble in A,B,C,D,A,... so there is a beautiful pattern of diagonal dots.  Others who are more imaginative will turn the answer sheet sideways and will try to spell out a message such as THIS TEST SUCKS with the dots.

     In Texas students get it in their head that the only test that "matters" is the tests they take as Juniors.  That allows for a huge (still less than 10% ?)  percentage of students christmas treeing as Freshmen.  That number, from experience not official data, is cut in half during the sophomore tests.  But there are a LOT of students doing this and it effects the score schools receive for the tests.  When this happens throughout the state the end results from school to school are statistically relevant and so we can justify the scoring that is used by the state to grade the school.

     Now when you place the fact that a percentage of high schoolers WILL not take a test seriously if it "doesn't count."  And, you then make that test a four hour test where they have to read passages and write essays and the "give-a-darn" (GAD) factor drops to a minimum level.   When the GAD reaches a critically low level some interesting things can happen.

    Take, for example, the student who christmas treed the entire test answer book even though the book was for two different tests and the test had a line that said "STOP and turn your test into the administrator" after the first test.  What was he thinking?  He was thinking "I want to finish this test as soon as possible so I can get out of here."  Or the students who were discussing (after the test had ended) what they had written.  As I was telling them that they can not discuss the test I heard one say that they had written Taylor Swift lyrics as their essay.  What they were doing was imaginatively writing essays that meant something to them - no matter what the prompt was about. (But it DOES make you wonder what the prompt was - was it really that bad or did they never even read it?)

     We have pretty darn good students.  They perform at a high level when they are asked and when the result of their efforts is meaningful to them.  If our school could be graded by the results of tests on every subject except math we would have received the highest rating given to schools in Texas in 2011.  These kids know how to take standardized tests and they know how to do well on them.

     So, how much did it "cost" to take the field test?  Well there were tests and answer booklets printed for our entire Junior class.  The Juniors lost about 5 or 6 hours of instructional time over two separate days.  Teachers were out of their classrooms monitoring the testing and administrators were involved with testing instead of being available for students, parents, and teachers.  Now that the tests have been given they will have to be graded - how much of the contract is taken up by that cost?

     The entirety of the cost can only be estimated. But the bottom line is that whatever dollar amount you want to assign to this field test has to be evaluated towards the validity of the test results.  I would estimate that between 20 and 40% of our students ended up not completing the test in a serious fashion. I would also be bold enough to say that another 10 to 20% wanted to just get it over as fast as possible. The end result is I can safely say that at least 50% of the students gave less than their full effort on that field test.  So how relevant will the data be from this test?  Not relevant at all and that is money that has been totally wasted.

What to do With Textbooks?

http://www.flickr.com/photos/marquette/

I haven't used a text book in my classroom in 4 years.  There, I've said it.  I did have a dozen copies of an Algebra 1 textbook in my class two years ago so the students could use it as a reference.  I love having copies of teachers editions of several publisher's textbooks for math at each level from Pre-Algebra to Advanced College Algebra in my room.  I also have copies of Algebra for Dummies and other "self help" math books that students may choose to use.

Today I took a sick day so I could work on getting over my bronchitis and I happened upon a discussion on Twitter - #beyondthetextbook.  That's one of the many things I love about twitter - I was able to sit at my laptop and cough my brains out without disturbing anyone (other than my wife and cats) while following an active, interesting, and relevant discussion.

As a teacher it is my responsibility to guide my students to finding the best resources on a topic.  And, once they have chosen resources teachers should be guiding the evaluation of the resources.  And, finally, teachers should be there to provide insight and understanding of the information found within the resource.  If the resource my students choose is a textbook then that is fantastic - as long as there isn't something else that explains the information in a better way.  If there is a better resource, then it is my responsibility (the evaluation of the resource) to guide them to this other media.

And so the things I kept seeing seemed to be avoiding the fact that a textbook is just a reference source.  I saw lots of things about going hi-tech with textbooks and having students create textbooks but I think too much time was spent on finding ways to replace the textbook with another form of something that is still, basically, a textbook.  IT IS A REFERENCE BOOK PEOPLE!  There I feel better now.

One of the better posts I read on this was by Doug Belshaw.   Doug went so far as to say that "the textbook is a symptom of a problem around assessment," in a tweet.   You should take time to read his post, read the comments, and read any of the posts he references in his post.

I live in a world where students research a problem.  They need to find and evaluate resources that help them find solutions to the problem.  If one of those resources is considered a textbook then, who cares? Notice I have not said anything about the political minefield of state textbook adoption.  You can't pay me enough to walk through there.  I found out a few years ago that textbook adoption committees mean nothing in my state and I'm sure they don't mean much in many other states who have textbook publishers in the lobbying business in their state.  Wait!  I said I wouldn't go there and I won't go any farther....

5 Quick Reasons to "Do PBL"

http://www.flickr.com/photos/photoman_sc/

It was like shooting ducks on the pond as my Grandfather used to say.  I walked in turned on Tweetchat to get ready for the chat on pbl (#pblchat) and there it was.  "MrsIsbellsClass RT @PBLChat: Anyone? Reply to=> @gavhays: Why do pbl? #parrapbl #PBLChat " 


   How could I resist that?  So I sent 3 tweets with a total of 5 answers to the question. 


 1.  Students more engaged = students less cranky  -   As students work on their projects they can really immerse themselves in their work.  This leads to a much more friendly and work-oriented atmosphere.  It's pretty nice. 


 2.  Teachers learn more because they learn with their students - quite often (at the high school level anyway) students will start researching a topic and they will get to a level that isn't immediately at the grasp of the teacher.  Sure, with a little brush up (usually), the teacher can quickly get up to speed and interject informative comments.  But when the student feels that they are learning with their teacher it brings learning to a new level. 


 3.  Students get to demonstrate their creative side - in almost every project there is some sort of end product that is shown to an expert panel or their classmates.  This might be a video, a poem, something actually built to demonstrate a process, or other item that was created with a students imagination and expertise. 


 4.  Students must face a panel of experts = must know their stuff - We don't always do a good job of getting outside panels but when we do the students really rise to the occasion.  Even when the panel is just a set of parents or teachers it means a lot to the kids and they prepare for their presentations accordingly. 


 5.  Teachers get to see their students grow with confidence and intelligence - It is really a wonderful feeling to see that shy student stand up and sing a song they have created to tell a story for their final presentation.  Or to see a student show the entire class that, yes, they do understand that very hard concept (that most kids figured he or she had gotten the rest of his group members to do while he sat back and did nothing.)   


      Those 5 reasons are reason enough but there are many more reasons to do PBL in your classroom.  If you ever want to see more come join us for #PBLCHAT on Tuesday nights at 9PM EST.  You might just be amazed at what you read.

The Post was originally published in December 2009

Assessment in a PBI Environment? How? 

     Whenever I talk with someone who is considering using Project Based Instruction (PBI) in their classroom I can be assured that the topic of assessment will be one of the first questions asked. Assessment, both formative and summative, is a topic that is guaranteed to bring out opinions from educators and this is magnified by teachers trying to visualize assessment in a PBI classroom.

So, what does assessment look like in my classroom? Well, we have tests and quizzes and daily "warm-ups" and written explanations and.......
Wait, that sounds just like a regular classroom? YES! That is exactly the point I am trying to make. Even though we are considered "100 % PBI" does not mean that we give up assessing the learning of our students.

There are a couple of items that we do assess differently and that, really, is all that we need to discuss. If you have options, in your bag of tricks, of assessing the learning that is going on in your class, then you are already at a point where you are ready to be a PBI teacher.

Some things that WILL look different or that you don't normally see in a conventional classroom include: Project Grades, Collaboration Grades, Oral Presentation Grades, Critical Thinking Grades, or even Group Assessment Grades.

PROJECT GRADES - A project is more than just putting together a pretty poster or a power point explaining some research that has been conducted. There needs to be definite demarcations for the beginning or introduction to a new concept or idea; a middle where research, hands-on learning, or direct instruction is going on; and an ending where the new concept or idea is presented by the students to show their level of knowledge of the material.

At each stage of this process there are chances to assess knowledge. There might be a pre-assessment to confirm the knowledge levels of each student during the introduction phase. During the middle stages there can be written quizzes, (or even tests for larger, longer lasting projects), there can be oral quizzes, there can be quick "knowledge checks" taken through clicker systems or surveys. We collect "Warm-ups" at the beginning of class or "Tickets Out" at the end of class as another way to check for understanding. Your imagination is the only limiting tool.


COLLABORATION GRADES - While we are walking around monitoring group work, we have an intuitive "grade" of individual efforts in each group. To put a numerical value to what we already know we can assign collaboration grades at the midpoint and at the end of the project. This might be done with a survey or Google Forms where each student gives the other students and themselves a grade (1 - 10, 1 - 5, etc.). Then you add them all up and determine the grade for each student. Don't worry about equity, students are both ruthless (about the other group members) and honest (about themselves). A fun way to do this is to give the groups a total score that the individual scores must add up to. Just make sure the total isn't evenly divisible by the number of students in the group.

ORAL PRESENTATION GRADES - Whenever we can, we make sure that the students are presenting their projects to an audience. They have worked hard and deserve to show off their new-found knowledge. The audience might be a panel of experts from a business, education majors from the local college or university, parents of students, other students, or whatever combination of people you want to come up with. We often hear "I haven't touched algebra in 20 years," or " I was never good at math." But these panelists don't have to grade on content, they can grade for how the students presented: did they have good rapport with the audience? did they have interesting videos/podcasts/powerpoint slides/etc.? did one member of a group dominate the presentation or were they equal partners? Just list the objectives you would like graded and let them do the rest.

Assessment, as should be no surprise, is done in a PBI classroom just like a conventional classroom. I hope this short post has dispelled any rumors that PBI means we don't have to do assessment. That rumor needs to go into the same category of unfounded myths as the one about PBI can't have direct teaching. Don't get me started on that one. Or, maybe that's a topic for another post.

What A PBL Teacher Does on Vacation

http://www.flickr.com/photos/uofdlibrary/ 

     Today is the first day of my Spring Break and I am relaxing by watching NCAA Basketball, keeping tabs on SXSWi, thinking about how I will be mentoring a school doing a school-wide pbl, and planning my classes for when the break ends.

     There are teachers who will not do anything teaching related.  There are teachers who will travel and will not allow school to enter their minds.  And, there are teachers, like me, who other teachers will accuse of having no life.

     To me you can never really leave the classroom too far behind.  As a pbl teacher I am accustomed to reflecting on how projects are going and what, if anything, will need to change to make the learning successful.  And that's where I sit today.  The project I am leading in my freshmen class is not going exactly the way I wanted and we, for the first time, have a true client to report to.  But my students don't really have an understanding of what is required for this to be a success.

     In my junior class my students have checked out and are not focusing on what they need to do to be successful in their project.  The problem with this group of students is that they created the project.  They asked to do the project.  But they have not embraced actually doing the project and that needs to change.

     I teach two different classes with two unique problems.  My freshmen are in a class they are required to take and they will respond to whatever I ask of them with a minimum of resignation that the work has to happen.  My juniors are in an elective that all but one or two are in reluctantly.  They do not want to learn about  digital electronics and the maximum I can expect is the minimum needed to pass.

     I look at the coming week away from the classroom as a chance to regroup with both classes.  I will take the time to create new projects that can be launched as soon as needed after the break.  With my juniors I may decide that it means a total regroup immediately upon returning to class.  With my Freshmen I know that it will be a minimum of 2 weeks before I launch the next project but I can find ways to make the wrap up of this project go smoother.

    An additional element in my planning is that we are moving into the standardized test season.  I can expect tests during every week between the week we get back and the middle of May.  Some weeks will only effect the freshmen and some weeks will only effect the juniors.  Other weeks will effect both groups of students.  These testing breaks can really effect the flow of a project and must be planned for up-front.

     So I'm on Spring Break.  So what.  I've got a lot of work to do.  And the work I do this week will keep  the months of April and May that much more enjoyable.  I'll get my break in June.  Then I just have to start planning for the 2012-2013 school year.  Or should I start planning that now?

Thoughts on SXSWedu

SXSWEdu 2012 by @cfanch

     Yesterday was my first day at this year's South By Southwest Education (SXSWedu) conference in Austin, TX.  I spoke twice at its maiden conference last year and was wondering where it would go this year.

     It appears that SXSW has figured out a direction they are heading.  Last year it felt very awkward being here - was it supposed to be a TEA (Texas Education Association) conference, a vendor conference, or an educator's conference?  We never really knew.

     This year there is a definite feel that we are seeing this conference embrace edtech and education entrepreneurship.   And that's OK.  If they continue in this direction it would be great to see ways to get more educators in attendance.  I attended a session with about 40 people and when the panel asked how many were teachers there were two of us.  There was another higher ed person in the room but the rest were all from industry.

    As we get more and more startups creating products that can be used in the classroom, we need to have educators here to hold conversations with these company representatives.  An app or program is only as good as its usability.  Will teachers want to use the product?  Will students use the product?  And, most importantly, HOW will they use the product?

    I have met many startups as well at this conference.  One of the things at SXSWEdu is LAUNCHedu where startups/edtech companies can hawk their wares and attendees can ask questions.  This is a great idea and I see wonderful opportunities for future South By's in this arena.

     Finally, this evening, I was able to attend meetups hosted by many of the larger edtech companies at the conference and it was nice to be able to talk directly to their representatives about the conference and how it can be improved.  Then many of us were able to attend a party hosted by Tech & Learning which really made us feel like we were more than just "educators."  These type of parties are prevalent at the regular SXSW conference and it really gave a nice ending to a great day.

     SXSWedu will be in its 3rd year next year and I'm hoping that they find a way of getting more teachers to the conference.  I was able to talk with WeAreTeachers founder Sandy Fivecoat and she came up with a great idea of holding a contest on her site that would allow a certain number of teachers to win scholarships to next year's SXSWedu and I'm hoping that can become a reality.

     Over and over I was asked for my thoughts on the conference and without hesitating I was able to say that it was MUCH better than last year.  There was good interaction in most of the sessions I attended and presenters wanted to know where the teachers were.  I would like to see more interactive sessions with conversations and hands-on opportunities but SXSWedu is heading in the right direction and I look forward to what it looks like next year.

   

There's a Thief Amongst Us

http://www.flickr.com/photos/fotocheska/2696700425/

You feel violated.  Your personal space has been intruded upon and you feel more vulnerable.  It's someone in your "family," or at least that's what you call all of these people who you live and work with. That's right I'm talking about how you feel when something comes up missing on board a Navy ship.

You trust your life with these people.  You work shoulder-to-shoulder for hours a day, 7 days a week, for months on end with these people.  And, one of them is a thief.

There really is no one lower than a thief in the military.  Yet, they are there.  You know it and you protect your things.  You never leave things unlocked.  You make sure valuables are on you or in a safe.

When I went into teaching 20 years ago I knew that there were students coming to me with very little and I knew there were students coming from very affluent backgrounds.   And, just like the Navy, these kids can be thieves.  I "knew" it but I didn't believe it.  I was wrong.

In every school I have worked I have "lost" things off of my desk.  Usually it's something small like a favorite pen or pencil.  Sometimes it's a little bigger and a little more important like a trinket from some event I've been to or from someplace I've visited.

Now, you have to understand that I have NEVER been at a safer school than I am at right now.  There really is a unique culture of family.  IPods and phones routinely are lost and then found and returned to their rightful owners.  Money is found and returned.  It really is an incredible school and I feel very fortunate to be working here.

But right now we have a thief (or thieves) among us.  Most of our students mature more rapidly than any others I have seen in my 20 years as an educator.  But there are still a few who are very immature and have a total disregard for our culture and our school image.

So what do we do about this problem?  Hopefully the current rash of thievery will lead to the apprehension of those with light fingers.  Hopefully the student body will be as disgusted with these kids as the rest of us are and will take it upon themselves to do something about it.  Even though this post started out very negative I need to say, again, that our school is incredibly safe and nearly 100% of our students "do the right thing."

And, just like the Navy which I love, there are some bad apples.  The bad apples are part of our family. They are the students in our classroom.  They work with other students in groups.  They eat with the other students at lunch.  They are bad apples but as the song goes, "one bad apple don't spoil the whole the whole bunch..."  We have a bunch of incredible students.  We have an incredible family.   Our family just needs to remember to lock up their things and keep our thieves away from temptation.