learning

Thank you, Dan!

We were lucky to have Daniel Furr join us this summer for the Grockit Graduate Research Internship program. Dan is a Ph.D. student in the University of California Berkeley’s Graduate School of Education, with a focus on Quantitative Methods and Evaluation.

At Grockit, Dan has been working on a variety of assessment-oriented projects, such as: on understanding how much data is needed to make use of an item response theory model in a Grockit network, on longitudinal models that help us refine our understanding of student improvement over time, and on extending our current models to incorporate the effect of collaboration, discussion, and repetition on a student’s probability of response accuracy. Understanding the effect of factors like these allows us to better select appropriate challenges for students in Grockit games.

About this internship, Dan writes:

Over the summer I focused on comparing approaches to estimating IRT models and experimenting with longitudinal models to measure user improvement over time. I compared estimations conducted with different R packages, with varying subsets of data, and ways of incorporating information on social interactions that occur alongside item responses. I used random item models to assess learning over time–time as calendar time, as item presentation order, and as discrete “sittings”. The people of Grockit are welcoming and innovative, and I feel fortunate to have had the chance to work with them and grapple with many interesting complexities.

I enjoyed working with Dan over the past few months, and wish him well on his return to academia in the Fall. Keep an eye on this blog for announcements of new assessment-oriented features on Grockit, several of which have benefited from Dan’s hard work this summer.

  • Del.icio.us
  • Digg
  • Facebook
  • Google Bookmark
  • StumbleUpon
  • Technorati

Being intelligent about supporting learning


As you probably know, Grockit offers learners the choice of three different modes of study: individual practice, peer study groups, and instructor-led lessons. What you might not know (unless you’ve read about Research at Grockit) is that these three modes of learning draws on ongoing research in three corresponding subfields of study. Last week, I got up-to-speed on the state of the art in one of these areas, individual study, at the Tenth International Conference on Intelligent Tutoring Systems. I wanted to share a few highlights from the conference here.

On Day 1, I started by co-organizing a workshop with Erin Walker and Carolyn Penstein Rosé (both of Carnegie Mellon University) on Opportunities for intelligent and adaptive behavior in collaborative learning systems. It was a great opportunity to bring together a group of people actively pursuing research at the intersection of intelligent systems and collaborative learning. Here is a snippet from the workshop website.

Intelligent tutoring systems are generally designed to tailor instruction to the individual student, but this does not mean that ITS-guided learning must necessarily be a solitary activity. A variety of recent systems have demonstrated ways in which an adaptive learning environment can incorporate and benefit from the presence of multiple learners. Similarly, students using computer-supported collaborative learning systems have been shown to benefit from the introduction of adaptive support that targets the collaboration. In this workshop, we invite discussion and seek to explore ways in which the combination of collaborative and intelligent aspects of a system can benefit the learner by creating a more productive learning environment.

If you’re interested in learning more about this workshop, you can find the presenter list, rapid-fire slides, and full proceedings on the workshop website.

As the week continued, I got a chance to hear presentations on a wide variety of research projects. A few of the talks that I found interesting this year touched on these topics: deciding if and when to provide hints, identifying the moment of learning from patterns in data, incorporating teachers into the process of designing a system, automatically generating hints from patterns in past data, understanding the ways in which human tutors are adaptive, incorporating dialogue agents in peer collaborations, and modeling learning gains over time. You can see the full spectrum of work presented in the conference proceedings index.

Grockit was proud to be one of the sponsors of this year’s Intelligent Tutoring Systems conference.

  • Del.icio.us
  • Digg
  • Facebook
  • Google Bookmark
  • StumbleUpon
  • Technorati

Games+Learning+Society+Grockit

Two weeks ago, Grockit participated in the Games+Learning+Society conference. On the second day of the conference, I joined Jeramy Gatza, Curriculum Innovation Specialist on the Research and Discovery Team at Florida Virtual School, to discuss a recent project in which eight Algebra I classes at Florida Virtual School piloted the use of Grockit as a supplement to their standard course curriculum. The presentation, entitled Collaborative Learning Games in the Virtual Classroom: Piloting Grockit at Florida Virtual School, took an interactive form. Each participant had a laptop, which allowed us to ground our discussion of the social, motivational, and collaborative aspects of the platform in first-hand experience. Here’s a bit more about the discussion, taken from the presentation abstract:

A virtual school can offer a student the ability to complete a course on their own schedule, from any location. The challenge in providing a flexible, individualized learning environment is that students may feel disconnected from each other, and can miss the opportunity to learn from interactions with their peers. Multi-player online learning games may hold a solution. By providing a venue for learners to connect and interact, these games can extend the benefits of collaborative learning opportunities to the geographically-dispersed students in a virtual school.

…Our goal for this workshop is to share with participants both an intuitive sense and data-grounded evidence about how multi-player learning games, like those in Grockit, can help connect, motivate, and engage students who are geographically and socially isolated. The workshop will conclude with a group-wide discussion of other experiences with, and opportunities for, using game-based collaborations as a way to connect learners across the web.

Two days later, Jeramy and I again presented together, this time to discuss our work with a group of educators as part of the GLS Educator Symposium. This presentation was nominally grounded in work that I’ve published on how we go about deciding between synchronous and asynchronous interactions for the various components of Grockit’s collaborative learning activities (for more info, see the source paper and the asynchronous discussion thread on this work). The real heart of the session, however, was devoted to a series of interesting questions raised by participating educators about the real-world task of incorporating Grockit (or a system like it) into the classroom setting. Jeramy Gatza provided a very interesting perspective, responding to several questions based on his experiences using Grockit at Florida Virtual School.

While I was at the Games+Learning+Society conference, I saw/heard/participated in a number of excellent talks, tutorials, demos, and keynotes. It was exciting to see the incredible variety of ways in which researchers and educators have been incorporating games into learning and learning into games. I’m excited to bring these ideas back to Grockit, and use them as inspiration for our own internal brainstorming sessions on games and learning. Stay tuned for new announcements on this front…

  • Del.icio.us
  • Digg
  • Facebook
  • Google Bookmark
  • StumbleUpon
  • Technorati

Welcome To The First Virtual High School Where The Students Do The Teaching

Everyone is welcome. All races, and all shapes and sizes may participate. Yes, in fact you the students of the world can come from anywhere and take charge of your own learning. You can come to take responsibility for your own education. You can bring the wealth of knowledge that the hundreds of millions of you have inside you to help raise each other up. Rise up! The social-learning revolution will not be televised, it will be online in real-time.

You can be good at math, or good at english or not yet good at either. You can raise yourself and your friends up, raise your standards, raise your skills. You can help each other 24 hours a day without downloads or plugins and you can do it all for free.

Why free? Because we have a social responsibility to do it. We must provide the students of our world the tools to help each other. We must do it just as we must pave the roads to schools or build the school buildings in the first place. If our students are obligated to get educated then we are obligated to provide them the means to be able to help each other when there isn’t a teacher and help each other when there isn’t an open school building. NFL player Dhani Jones, a vocal advocate for youth responsibility, calls it ‘extracurricular learning’ and I agree that it’s important not because of society, or parents, but because of YOU the student. It’s important for you to extend your learning outside the classroom and to amplify it inside the classroom. It’s important for YOU the student to take responsibility for your own learning. So, to do our part, we are launching the Grockit Academy as an always-open, student-to-student, algorithm-enhanced, online High School. We’ve decided not to wait until the new school year so we’re starting it as a Summer Enrichment Academy where you can begin learning with your peers from all over the country and world, right here, right now.

We launched Grockit Academy and made it free for students to work with each other in live, multimedia, collaborative online study rooms because we have a social responsibility to do so BUT, it also makes good plain economic sense.

Let’s talk money. Keeping schools open and running just to cover the electricity, water and maintenance costs a lot more than running computer servers. Maintenance and operations in schools is about $500/student/year. It’s probably more, but let’s go with it. With 65M students in K12 that gives us a total of $32,500,000,000. That’s thirty two billion dollars. But that’s not it. We need teachers and we should pay them more than we do. We have about 3 million teachers at about $50K each so that gives us $150,000,000,000. Yes one hundred and fifty billion dollars. So, schools and teachers, the backbone of our educational system, the people and places that shape our lives, are not free. As awesome as it would be, we can’t afford to make schools and teachers available 24 hours a day because we would more than double the $120B+ we just mentioned. But neither can we afford to leave our students without the tools and resources to extend their learning after school and across the world and to each other. Servers and bandwidth are not free either but relative to hundreds of billions of dollars they are and they can run 24/7. The most compelling economic argument, however, is that students collaborating and helping each other is FREE and, not only is it free, but there is a compounding affect on learning because all students learn from the interaction. Sorry for all the math but it’s just too interesting.

It makes sense for students and learning.

Research has shown that small group study is the best form of instructional design. Ok, well then let’s do a bit more math. There are 65M students and 3M teachers. We can create all the small group study settings we want if we match students to students and they become more responsible for their own learning and their peers’. There aren’t, however, enough teachers to go around to make small groups for everyone. The smallest groups you can get with a teacher is about 20. In addition to their instructional design benefits, small group interaction through a computer based system allows for unmatchable data acquisition, performance report creation, data on learning over time and assessment of students and instructors. This data can then be put to use by adaptive algorithms that create even more efficiency and effectiveness in the learning eco-system by customizing the student’s and the group’s experience. This can’t be matched by any other system and this isn’t students working by themselves with a computer – we are talking about live real-time student to student engagement.

It makes sense for teachers.

What about the teachers you say? I say, set them free! Teachers will be the first to tell you that managing a classroom of 20-30 students is not how they would design things. When I was a teacher I had this exact problem too and small group study is how I learned to deal with it. When I can organize students into meaningful groups, provide them with some rules of engagement and hold them responsible for each other, student engagement improves drastically, quality control improves dramatically and I as a teacher am set free. I am free to facilitate the small group study interaction and actually free to provide more individual instruction for each student because all other students are engaged in learning while I interact with individual ones.

It makes sense for schools and society.

Why limit your student population to a few hundred when you can leverage the knowledge of hundreds of millions? In Grockit, students from all over the world collaborate everyday. They have answered millions of questions and exchanged millions of live discussion messages about math, english, and even the SAT. The number one asset any student has in school is their peers and it’s our job as instructional designers and administrators to facilitate that interaction, prepare students for a global world and leverage the knowledge already there in the millions of students around the world. By continuing to crowd source, filter, and raise the best questions, answers, explanations, and hints to the top, as a society we benefit from the accumulation and organization of pedagogical knowledge. This will make learning increasingly more effective and efficient and we will all benefit from it.

In Grockit Academy, our goal is to participate in this movement by continuing to provide collaborative group study for free. For premium features like adaptive solo practice, detailed reports on your performance, and unlimited access to premium content, we offer a $79 yearly subscription. This gives you access to the same assessment and algorithms that, until now, entire school districts have paid millions of dollars for. This gives you access to as much math and english content as you would have in over $500 worth of textbooks. And, textbooks don’t learn about you, don’t adapt to you, don’t interact with you, and don’t connect you to the millions of students studying the same thing as you. Grockit Academy puts the power of an entire school district right in the palm of your hand, or tip of your finger if you’re using an iPad.

Last week I saw President Clinton speak and he closed his speech by saying something really cool.

“We have to build up the positive and reduce the negative forces of interdependence.”

While it might sound esoteric, I think it’s just his way of saying that we have to put more resources and effort into the types of human to human interactions that create win-win outcomes. There are 65M students in K12 in the United States alone. In the world, the number of students is in the hundreds of millions. These students depend on each other everyday and will be running the world together in a couple of decades. There is a wealth of untapped knowledge in all of you and we are here to help the world teach itself. Welcome to Grockit Academy. Come on in, we’re always open.

  • Del.icio.us
  • Digg
  • Facebook
  • Google Bookmark
  • StumbleUpon
  • Technorati

Do students spend more time solving appropriate challenges?

When you play a solo game in Grockit, the questions that you see are selected based on your recent performance (Grockit gets more adaptive). Our goal in doing this is to offer you challenges that are neither too easy nor too hard for you, in order to provide better opportunities for learning.

Our concept of how difficult you’ll find a question is based on our estimate of the likelihood that you will answer the question correctly. If, for instance, we estimate that there is a 90% chance that you will answer the question correctly, we think that you’ll consider that question to be pretty easy. Others may find the same question difficult, but you’ll probably find it to be straightforward.

Today’s question: Does a subjective measure of difficulty have any relationship with the length of time it takes a student to solve the problem?

Think for a moment about your own problem-solving experiences. Do you think that you spend more time on a harder question than on an easier question? Less time?

We record a lot of interesting data when students answer questions on Grockit, on some of this data can help us answer the question at hand. Below are two plots of the average time taken by Grockit students to answer GMAT Quantitative and ACT Math questions, as a function of how likely we believe the student to correctly solve the question. When you look at the plots, the first thing to note is that the relationship between subjective difficulty and response time isn’t linear.

These plots indicates that, on average, students are spending less time answering each question that are subjectively very easy or very difficult than on the questions that are in between. Do you find this surprising? Interesting? Does this reflect your own experiences studying? How would you interpret these plots?

  • Del.icio.us
  • Digg
  • Facebook
  • Google Bookmark
  • StumbleUpon
  • Technorati

Ongoing research at Grockit

We’re really excited to share some of the research that we’ve incorporated into making Grockit the learning tool that it is. We just added a new Research at Grockit page to the site, where you can learn about how Grockit continues to participate in, draw on, and contribute to the research community.

Along these lines, I’ll be co-organizing a workshop on Opportunities for intelligent and adaptive behavior in collaborative learning systems at the Tenth International Conference on Intelligent Tutoring Systems in mid-June. The workshop website is now online, and accepted papers will be posted in June.

Keep an eye on the Research at Grockit page, as we’ll be adding more to it over the next few weeks…

  • Del.icio.us
  • Digg
  • Facebook
  • Google Bookmark
  • StumbleUpon
  • Technorati

Making Grockit more adaptive

I’m happy to announce a recent improvement to a popular Grockit feature: In solo practice games, the questions that you see are selected based on your recent past performance. Grockit’s tailored Challenges — short assessment-oriented games that are unlocked over time — have a been popular way for students to benchmark their performance and get feedback on where they stand. We’ve now incorporated the core algorithm used to create Challenges into all solo practice games. Questions posed during these games are now selected in a way that takes into account both what you have seen in the past and how you’ve been doing on recent questions. We know that different students are stronger in different parts of a test, so you may notice that you’re seeing harder questions in one section than in another. You’ll find that the personalized question loader takes these variations into account.

As always, we’d love to hear your feedback. Please feel free to leave a send us a email or leave a comment below.

  • Del.icio.us
  • Digg
  • Facebook
  • Google Bookmark
  • StumbleUpon
  • Technorati

Reimagining learning

Just a quick post to let you know that several hundred great new ideas on how we might reimagine learning have just been made public, spurred by the Digital Media and Learning competition. Not only can you read about each of these ideas, but — this week only — you can contribute to the conversation by adding your own comments. Be sure to check out our idea, which is about how we can leverage Grockit’s platform for live collaboration to create a new game that actively engages learners in asking questions and sharing their work with the world (through Grockit and Connexions). We’d love to hear your feedback, so please leave us a comment!

  • Del.icio.us
  • Digg
  • Facebook
  • Google Bookmark
  • StumbleUpon
  • Technorati

Grockit Questions Are Tailored for You!

Here at Grockit, our philosophy is that students learn best when challenged with problems of appropriate difficulty. Each student has a unique toolkit of complex reasoning, quantitative and English language skills, and Grockit’s analytical software provides that student with feedback on their performance, their progress, and their strengths and weaknesses. This feedback enables Grockit students to tailor their practice and allocate their study time more efficiently.

Grockit’s ever-growing bank of unique questions has been written and reviewed by expert instructors and seasoned content writers. We design our questions using College Board, ACT, GMAC® and ETS® released questions from previous exams, along with other specially-selected resources. This allows us to best model actual questions that you will see on your test day. Each question is characterized by its difficulty level and the specific skills that it tests, and we use that information to provide you with fine-grained feedback on your performance and learning. When combined with the data that we’ve collected from your recent performance, this meta-data helps us provide Challenges custom-built for you.

50th and 90th Percentile students alike will benefit from Grockit’s algorithms and incremental learning platform. We aim to challenge you with test-true practice questions to help prepare you for your test day. Good luck with your studies!

  • Del.icio.us
  • Digg
  • Facebook
  • Google Bookmark
  • StumbleUpon
  • Technorati

25 blogs

Many of the approaches that we draw on at Grockitadaptive web-based learning, study group formation, embedded assessment, computer support for collaborative learning, educational applications of data mining and social network analysis, learning in games, and the benefits of practice, feedback, spacing, teaching, discussing — are no longer exclusively discussed in academic journals and conference proceedings. Several of the researchers and practitioners in these areas have hit the blogosphere, where ideas are informally sketched, quickly shared, and freely accessed. Here are 25 such blogs that are currently in my newsreader:

OLDaily
Learning Games Network
Digital Media and Learning
Epistemic Games
Clark Aldrich On Simulations and Serious Games
Confessions of an Aca-Fan
Raph Koster’s website
Learnlets
Will at Work Learning
Informal Learning Blog
CSCL Community
Connectivism: networked and social learning
P2P Foundation
Iterating Towards Openness
Sharing Nicely
TravelinEdMan
Innovate journal
Learning Sciences and Educational Technology
International Review of Research in Open and Distance Learning
Infinite Thinking Machine
Online Sapiens
Elearnspace
Educational Technology News
Fortnightly Mailing
Virtual Canuck

Have some that we’re missing? Leave a link, and we’ll check it out.

  • Del.icio.us
  • Digg
  • Facebook
  • Google Bookmark
  • StumbleUpon
  • Technorati