I want to start off by making this post’s intention clear: my goal is to level the information playing field which will enable us to improve math education conferences for speakers and attendees. Working together we can create something better than any one of us could have created on our own. A key component of this philosophy is that everyone has equal access to information.
I am not trying to imply that any one conference is better or worse than another. I think that each conference has attributes worth considering when reflecting on how conferences could improve.
With that in mind, I have collected data from the 2014 California Math Council (CMC) South conference, a 2015 National Council of Teachers of Mathematics (NCTM) Regional conference, and the 2015 Twitter Math Camp (TMC) conference. Many thanks to the great people who organize these conferences. They put in significant work to make these conferences run smoothly and I appreciate their help in consolidating this data. Below the data I share my observations in what I hope is an objective and balanced manner.
| Conference | CMC South | NCTM Regional | TMC |
|---|---|---|---|
| Locations | Palm Springs, CA | Nashville, TN (venues change) | Claremont, CA (venue changes) |
| # of Days | 2 | 3 | 4 |
| How old is this conference? | 55th annual | 52nd annual | 4th annual |
| # of attendees | ~3500 | ~3000 | 200 |
| # of speaker applications | 429 | ~800 | 63 |
| # of speaker acceptances | 337 | 258 | 63 |
| % of speakers accepted | 78.6% | 32.3% | 100% |
| % of attendees that are speakers | 9.6% | 8.6% | 31.5% |
| Is a rubric used to evaluate the speaker proposals? |
No | Yes | No |
| Is prior speaker data used as an acceptance factor? |
Yes | No | No |
| Registration fee | $155 | $310 to $233 for teachers who are NCTM members |
$0 |
| Speaker discount | $105 (for primary speaker and also for one co-speaker) plus $400 to $500 for travel related expenses |
$310 to $233 (primary speakers receive free registration) |
$0 (already free) |
CMC South is held in Palm Springs, CA every year. NCTM holds its regional and annual conferences in various locations across the northern, southern, western, and eastern regions of the United States. TMC holds its conference in various locations across the United States.
Number of Attendees
CMC South and NCTM regional conferences’ attendance varies but has ranged from ~3000 to ~5000 attendees. There is no limit to the number of attendees that may register for this conference. TMC’s attendance has grown each year and is primarily limited by the number of available sessions for attendees (which is limited by the number of speakers they have).
Speaker Acceptance
In 2015, TMC accepted 100% of the speakers that applied. In part, this speaks towards the reality that a conference needs to have enough speakers so that it has sessions for all its attendees. CMC South and NCTM have many more speakers apply than there are spots, so all are not accepted.
Using Prior Speaker Data
Only CMC South uses prior speaking data as a criterion for speaker proposal acceptance. The 2014 CMC South conference collected two types of data: speaker evaluation data as submitted by attendees and room attendee counts, both towards the beginning of the session and towards the end.
Unfortunately, I don’t have access to the CMC South evaluation data. However, based on conversations I have had with people who have seen the complete data set, it appears that most attendees have a tendency to leave very positive reviews such that while it may be easy to distinguish the few bad sessions from the good sessions, separating the average sessions from the good and great sessions proves more challenging.
Room attendance can be measured in terms of total attendance or % of attendance that stays for the entire session. Factors that skew total attendance include the session’s room size (small rooms don’t hold many people), the session’s time slot (early or very late sessions are often poorly attended), the session’s location (many people skip sessions that are too far away), or the session being at the same time as a major speaker. Factors that skew the percentage of people who stay for the whole session include overlapping sessions (some people leave their current session early to go to the next session that starts during the middle of the session they are currently at) and the next session’s location (people may leave early if there is not enough time between sessions to get to the next session).
Rubrics
NCTM began using rubrics to evaluate speaker proposals somewhere around 2008.
Fees and Discounts
Registration fees and speaker discounts are largely influenced by the degree to which volunteers run the conference, venue costs, and the quantity of work being done. CMC South and TMC each run one conference per year and are volunteer run while NCTM runs four conferences each year and has many full time employees to run its operations. CMC South and NCTM have to pay for its large venues whereas TMC has been able to secure smaller, free venues.
Thanks Robert for collecting, organizing and sharing this information.
The area that interests me is the prior speaking data. At NCTM, as we have been thinking/talking about about the evaluations/feedback component as part of larger Professional Development/Conferences strategic planning effort.
You are correct that NCTM does not formally collect data about each session/speaker. We do ask attendees to point out sessions that stick out in positive way as well as those that we should be concerned about. I think it can provide staff with a potential red flags as we look to new programs, but clearly not what at the level of specificity needed to make session by session decisions. (In the end it may have similar impact as the CMC efforts as you described them.)
The discussions are part of a larger effort so they have a way to go but one thing that is clear to me is that this process needs to span the entire process, and be well communicated and well connected to be meaningful. One process we learned about was something like the following:
– Proposal submissions include specific goals for attendees as required content.
– Training available to proposers to understand the system.
– Each session’s goals are included in program materials
– Training available to presenters to understand the process and expectations and agreement from presenters to follow process.
– Presenter (or presider) are required to show and discuss goals of session within the first 10 minutes of the session.
– Presenter (or presider) undertake written feedback from attendees during the last 5 minutes of session with session goals displayed
– Attendees asked to provide feedback on the achievement of the specific session goals.
– Information accumulated immediately, consolidated and used to provide direct feedback to presenters to improve their sessions (potentially onsite.)
– Consistent disregard for process and goals or lack of improvement results in removal from program consideration.
– Information on presenters is integrated into future program panels
Again, this is an example and part of larger strategic planning discussion but helps us frame how a process might play out. It is clear to me that meaningful collection and use of session/speaker data would impact many aspects of the program review and even the conference.
Appreciate others perspective on this as well.
Dave
Thanks Dave. Coming back from the 2015 NCTM Annual Conference in Boston, I don’t recall any way to give the good and bad feedback you describe. So, at the very least it could be more publicized. I completely agree with your statement of “this process needs to span the entire process, and be well communicated and well connected to be meaningful” is very true.
I appreciate you sharing your list of ideas. There are some that I may have different opinions on, but I think it is a great start. Thanks again.
Robert and Ihor, I appreciate the dialogue.
As to the feedback, NCTM sends a follow-up survey directly to all who attended the Annual, excluding exhibitors and guests, that includes two specific questions on sessions attended. Participants are asked to identify any sessions or speakers that exceeded their expectations and why, and a following questions to identify any sessions that did not meet your expectation and why. The response rate is in the neighborhood of 20%.
Ihor, as you clearly know developing a program is complicated process. When I spoke of “future program panels” I was referring to the program committees that develop the program. I agree it should be more than who you know is a good speaker. The development, transformation and connection of evaluation and selection processes is a transformational process. This undertaking changes from a isolated focus on an individual event to creating a foundation that spans and informs all professional development offerings. In the end it will ask more of speakers and participants to play a role in the process in collaboration with program development teams.
Thanks Dave. Perhaps in time, and with technology, there will be a way for NCTM to implement more granular, real-time session feedback collection system. Other conferences, such as CMC South, get session specific feedback at much higher rates than 20%.
Hi Dave,
When I was on the program committee in 1999 I was concerned about the quality of the talks because most of the criteria for speaking was about appropriateness of topic and not much on the experience or quality of the speaker. So it was a crap shoot whether the speaker was good or not. I’ve noticed Dan Meyer’s emergence as a key player on the NCTM scene and it’s well deserved. But I still worry that the best speakers are not selected because the criteria for selection has not changed a whole lot since my day on the committee. Your last point
– Information on presenters is integrated into future program panels
not just panels, but all sessions! I realize that it requires some good analysis to do this, but shouldn’t groups of math teachers on committees be good at that?
BTW-One person who should be invited to speak again is Robert Kaplinsky. I was at his presentation and it was excellent! 🙂
-Ihor
Ihor, thanks for your feedback and kind words. Clearly we are reaching a point where technology is making it easier to acquire data. I agree that all sessions should be receiving data. The point isn’t necessarily to shut out speakers who may need more experience, but certainly it should make it easier for great speakers to get accepted again. If you have any thoughts on what data to collect and how to account for potential skewing, it would be be much appreciated to help move the conversation along.