Mental models to support competence in computer programming

Mental models

These are the mind’s ‘mechanisms’ for explaining and predicting phenomena. I have written more in my PhD, and you can find good expansions of the idea in Michael Simmons’ post or in the Wikipedia article on Mental Model The idea originated with Craik in World War II and elaborated by Johnson-Laird amongst others, all of whose work made me interested in developing computational models(!) of mental models – a subject of interest to me in the late eighties when I was a member of the London Mental Models group.

I do not think this is the same thing as modelling phenomena in mathematics or in computer programs – such simulation models are expressions in external languages, unlike mental models, which are mostly private to our minds: interconnected, fluid, faulty and ultimately unknowable. When we create and communicate external expressions in natural or formal language, this leads to the possibility of proof, execution and formal reasoning in a shared world of knowledge. But these external expressions aren’t mental models: they are in a linguistic form that can be interpreted by others and in formal cases, by machines.

Nevertheless I believe the unknowable internal mental model remains a useful notion when we think about designing effective learning. I apply the notion here to the design thinking needed for effective courses, materials, pedagogy, software, assessment to teach programming.

Mental models in Computer Programming

This is my second draft diagram which represents ~~four~~ five key mental models that a learner must develop (and continue to develop) as they increase competence in programming. It has been much improved after listening (and reacting) to Jane Waite talk about abstraction in programming. I have now made a poster – Mental models to support competence in computer programming – for the London Computing Education Research Symposium on June 11th 2018.

Programming - five areas of mental model — Programming – five areas of mental model

Problem Comprehension

This mental model allows the learner to reason about the problem itself – it may develop as the learner combines problem solving and design to make a solution. Sometimes prior knowledge can help; for example, Papert would argue that children enjoy, are competent and have mental models about the way their body can move in the physical world. If a problem is aligned to such competence, they can more effectively debug their program (body syntonic) and feel engaged with the challenge (ego syntonic).

Programming Language

This mental model is about the parts of the language – the distinctions between different linguistic components and their connection to create programs. Scratch supports this mental model by categorising statements and thus offers recognition rather than recall. It also reinforces appropriate syntactical combinations, so that the focus is on their meaning, in isolation and in combination.

Notional Machine

The notional machine is a mental model concerned with the variables, computer memory (for data and program), ‘program counter’ – a hidden variable that determines which statement is executed next and thus flow of control. It is much more complex with Scratch than in the past, since multiple parallel process are readily designed using sprites. The design of solutions in this way can be quite different from that made with single process thread programming, but makes the mental model challenging. My favourite example of this was the solution I developed using Scratch with three sprites to draw lines to fill in a shape, hoping to use it with TurtleStitch to make embroidery. Then I discovered that TurtleStitch (a Snap derivative) had only one sprite – my solution was useless due to a mismatch of my mental model of notional machine for Turtlestich and Scratch.

Microworld / Domain

The microworld is the concept of a limited ‘space’, designed to suit a particular class of problems and usually with an ‘object to think with’. The turtle geometry microworld is the most famous, but not the first in Logo. Before that came sentence construction using lists of words to manufacture amusing nonsense! Making a mental model of a microworld’s affordances allows the learner to map solutions to problems and to relate to the notional machine and the programming language, within a limited but meaningful domain.

Interactive Development Environment

The mental model here is of a complex user interface to understand and write programs, manage program files, debug programs and produce results. Sometimes it spans several computer applications, such as an editor, file manager and version control, and sometimes it csn be combined in one place, as with Scratch. It is the interactive development environment (IDE) which can help or hinders the user in forming the mental models of programming language, notional machine and microworld through visualisation and interactivity.

Connections between mental models

The greatest challenge to the learning programmer is in the connections and overlaps between each of the mental models. I believe the educational designer (especially the designer of the IDE) must pay attention to each of these and the questions in the diagram are to help the designer think about that.

13 December 201713 December 2017

The Learner at the Centre of Educational Design

These are the slides for a talk I gave at the University of Hyderabad back in January 2017, after an invitation as Visiting Professor there to discuss the design of a new Education Department and Masters Programme. They were particularly keen to solve problems relating to the legacy of the caste system and how it affects education. I offered my PhD theses, and spent time elaborating them, including this one offering an analytical framework for educational design which took a learner-centred view to inform the process.

Educational design is often, quite naturally, made to fit the organisational needs of the institutions and society providing education. The analysis presented here is intended to support the designer in taking a learner-centred approach, echoing the work of Donald Norman in user centred design. By posing eight questions that we imagine a learner may need to answer in order to have a complete educational process, the design challenge can be broken down in order to identify where an educational design is at fault, which in turn acts a focus for creativity and development. This talk describes this analysis and outlines each question’s detail.

Here is a printable poster of the analysis: The Learner at the Centre of educational design

28 October 201728 October 2017

Knock knock! – an interpretation of ‘body syntonic’

I recently worked with colleagues to offer similar workshops at two conferences – SCoTENS 2017 in Dundalk (with Pamela Cowan from Queens University Belfast and Elizabeth Oldham from Trinity College Dublin), Ireland and ATEE 2017 in Dubrovnik, Croatia (with Nina Bresnihan, Glenn Strong and Elizabeth Oldham, all from Trinity College Dublin).

The workshops introduced our ideas about using a version of paired programming to give confidence to novices in programming. We had developed these ideas, together with colleagues Mags Amond and Lisa Hegarty, also from Trinity College Dublin, through the CTwins project funded by Google’s CS4HS – Computer Science for High School.

The workshop slides for ATEE 2017 also included ‘Art’ in the title, since it was my notion that developing an art project would be personally fulfilling.

You can see how I have been a little pre-occupied with the relationship between art, craft and programming through my recent blogs:

In a happy co-incidence, I today found myself in a useful conversation about the design of the programming tool, Scratch, that we used in the project and the workshop. In the conversation, we rightly focussed on the design of Scratch, which has become so wildly popular that a heavy weight of responsibility lies on the development team to get it right. I tried to explain why Scratch is important in this blog post:

Jigsaw programming

Nevertheless, I feel that as well as considering the tool design, we must also shift attention to the activity and mental models that I believe learners symbiotically develop alongside their use of the Scratch tool. The Logo programming language developed in 1967 and its turtle geometry microworld is one of the most potent developments to recognise such activity and mental modelling – although I believe not the earliest (I believe sentence generation using lists preceded it?).

A microworld is a slippery concept, but Richard Noss and Celia Hoyles neatly sum up its importance in their book ‘Windows on Mathematical Meanings: Learning Cultures and Computers‘:

“In a microworld, the central technical actors are computational objects. The choice of such objects and the ways in which relationships between them are represented within the medium, are critical. Each object is a conceptual building block instantiated on the screen, which students may construct and reconstruct […]. To be effective, they must evoke something worthwhile in the learner, some rationale for wanting to explore with them, play with them, learn with them. they should evoke intuitions, current understandings and personal images – even preferences and pleasures. The primary difficulty facing learners in engaging directly with static formal systems concerns the gap between interaction within such systems and their existing experience: it is simply too great. That is why computational objects are an important intermediary in microworlds, precisely because the interaction with them stands a chance of connecting with existing knowledge and simultaneously points beyond it.”

In the turtle geometry microworld, the computational object is a robot turtle on a stage, equipped with a pen to trace out lines as it moves according to program steps.

Scratch starts with a different microworld sporting a cat rather than a turtle and is a particular kind of computer game with interacting sprites. It leaves in the jigsaw blocks for a turtle geometry microworld but they are somewhat spoiled by the sideways view of a stage rather than the top down view of the space that the turtle inhabits.

In the conference workshops we asked completely novice learners (adults using Scratch and ScratchJr) to program knock-knock jokes, with two sprites and message passing to synchronise the joke-telling activity.

Firstly, together with colleagues, we performed this joke (thanks to Pamela Cowan for such an excellent idea, performance and preparation):

Ghost: Knock knock!
Cat: Who’s there?
Ghost: Boo!
Cat: Boo who?
Ghost: No need to cry!

Secondly, we asked the adults to humanly perform their own jokes working in pairs, so that one adult would be the first actor in the joke and the other the second. I was building on the concept of ‘body syntonic’ which is so powerful in the turtle geometry microworld, but in this case, it is the act of interactive joke telling that forms the mental model of the problem, to be then expressed formally in programming and debugged.

In the Scratch turtle geometry microworld, the pen jigsaw blocks are the foundations of formally expressing the acts of an imagined turtle with its pen. Children (and adults) can ‘be’ the turtle and act out the actions either bodily or in their heads, exercising their mental model of the turtle, which may then help them debug their formal expressions in code (jigsaw blocks).

In the case of our knock-knock microworld, we presented on the projector screen a subset of jigsaw blocks to start with:

In one instance of the workshop, to my delight, one learner added other blocks, using repetition to tell a more complex joke.

So perhaps the set of immediately available jigsaw blocks should reflect the microworld the learner’s imagination and mental models are anticipated to inhabit? I would go further and propose microworld-appropriate stages (and stage views, as we have in Turtlestitch and Beetleblocks), sprites and costumes. In Turtlestitch I would propose a spider sprite/costume and indeed rename it Spiderart or some-such. Perhaps there should be a choice of microworld, “I’m doing turtle geometry today” which leads one to the set of jigsaw blocks most appropriate to that microworld? I emphatically do not mean that this means restricting access to the wider set of jigsaw blocks, simply that it provides the best recommendations from the menu for the kind of restaurant you want / are ready to eat in.

To extend an already overworked metaphor, after the learner has been eating at diverse restaurants, each founded in the same underlying elements of heat, ingredients and combination, perhaps they would begin to strengthen their knowledge of the invariates which inform the mental models that underly their understanding of notional machine and programming language?

20 October 201712 March 2023

Turtlestitching – programming embroidery

My first introduction to programming an embroidery machine came at the Scratch 2015 conference in Amsterdam, when Andrea Mayr-Stalder from Vienna presented the Turtlestitch programming environment based on Snap. I didn’t take that much notice, however lovely the designs and the possibilities were.

Then I went to Seymour Papert’s commemoration in Boston in January 2016 and met Susan Ettenheim from New York. Susan had joined forces with Andrea to explore Turtlestitch and was on a learning journey with Susan’s student Jennifer Lin. Jennifer was struggling with a problem – how to fill in space with an embroidery machine using Turtlestitch. The task she was attempting was to fill in a petal shape. Artemis Papert had made a good solution, which tackled the problem using variables and trigonometry. In this program, the petals have become leafs:

Artemis Papert's petal — Artemis Papert’s petal

Susan told me that Jennifer struggled at first to understand such sophisticated mathematics. Never one to ignore a challenge, I designed an alternate solution which stuck to ‘body syntonic’ principles – essentially, exploiting a learner’s prior knowledge of moving their own body to make and debug a computer program. My solution involved three ‘sprites’ – one to run around one side of the petal, one to run round the other and finally another to run between them, filling in the space. One can imagine children actually acting this out for real, collaboratively, as a precursor to programming a solution in code, in the same way that turtle geometry allows them to solve geometric problems by imagining they themselves are moving and turning. [UPDATE March 2023 – here is a video we made in 2019 to illustrate!] It is salutary to note that my solution involves synchronising concurrent processes – a topic I would have considered above my pay grade, let alone appropriate for learners as young as 5! (Later I found out that ScratchJr, designed for younger learners, also included this kind of notional machine!).

After this, I was hooked, and at Scratch 2017 in Bordeaux I met Andrea, Susan and Jennifer together with Michael Aschaeur, who had programmed Turtlestitch, and had the opportunity to talk about my ideas and learn how they planned to go forward. As a result, I resolved to buy an embroidery machine!

I purchased a Brother Innov-is F440E embroidery machine in September 2017 from SOSBrother in Bray, I resolved to create an opportunity to play with colleagues and friends using Turtlestich to explore programming and embroidery, and thus was born the Turtlestitching workshop held today, Thursday 19th October as part of EU Code Week.

The introduction that Susan from SOSBrother gave me in to the machine’s operation was invaluable and I tried to pass on all I could remember to my collaborators.

Mags created designs seeded by our date of birth:

https://twitter.com/twitter/statuses/921091656694235137

Mary Jo followed the brilliant Turtlestitch cards to create some lovely interlocking circles:

Mary Jo's simple but effective design — Mary Jo’s simple but effective design

Jake was like a duck to water, his work here modelled by Mags:

Jake's pattern — Jake’s pattern modelled by Mags

John’s design started black and white, but became really beautiful when using the multi-coloured thread:

Glenn inspected, analysed and modified an existing pattern to fit the Brother’s 18 by 13 cm space:

And after they all went away, I made my own Automated Landscape (the illustration at the start of this blog) into an embroidery, using the wonderful multi-coloured thread.

The workshop taught us several things:

“move 10 steps” produced a 2mm stitch, which was a ‘good’ size stitch;
going over patterns twice or even three times could make stronger designs;
multicoloured thread could make spectacular embroideries;
more time was spent discussing computational issues than embroidery issues;
it was hard fun!

I am absolutely delighted to announce that Trinity College Dublin’s Visual Arts and Performance scheme have agreed to fund a course and exhibition based on this, following a successful Wearable Electronics Workshop last year – look out for the advert in the New Year!

4 May 20177 May 2017

Wearable Electronics Workshop

This is the story of the Wearable Electronics Workshop, given by MAKESHOP by Science Gallery Dublin in March and April 2017, in collaboration with the School of Computer Science and Statistics, Trinity College Dublin.

In 2015, I was introduced to the idea of ‘pimping your badge’ at a conference by friend Mags Amond.

It involved adding a watch battery and an LED (light emitting diode) to my conference badge to make it light up – my first wearable electronics!

Conference badge with LED and watch battery

Mags was later to get involved in the workshop to introduce some basic ideas about circuits.

That Christmas, at my annual birthday party, I gave similar treatment to a bow tie, and it was well received.

The following summer (2016) I found myself in St Vincent’s hospital in Dublin, having my heart checked out when experiencing a rare irregular heartbeat (arrhythmia). The time in the hospital was short, since I proved to be in robust health, but enough to stimulate an idea for another wearable electronic – something driven by my own pulse:

The experience stimulated the following design, ‘LED by the heart’, which pleased me, so I resolved to try and make something wearable with it.

'LED by the heart' - a design based on the symbol for an LED (Light Emitting Diode) — ‘LED by the heart’ – a design based on the symbol for an LED (Light Emitting Diode)

Feeling a little worried that it might be hard work, I sought friends and advice. My good friend at Trinity College Dublin, Doireann Wallace, offered support and interest, and later helped me to invite all my party guests in Christmas 2016 to complete and wear a glowy wearable. Doireann kindly prepared for this by cutting felt into Christmas shapes and stitching battery pockets for the partygoers to assemble.

Felt fir trees for glowy Christmas badges

Friends wearing their glowies at the party

On a hunt for new components and advice, I visited the MAKESHOP by Science Gallery here in Dublin. There, I got into a conversation with Jessica Stanley, who runs electronics workshops for them, and as luck would have it, had a background in wearable electronics. She had wanted to offer a course on this for some time, so I promised to help by finding some other participants.

Then I came across the Trinity College Dublin Visual Arts and Performance Fund. I made an application, and the committee were kind enough to sponsor our course. So in March 2017, we recruited participants to join us: some had craft experience, others programming and design knowledge – all were keen to know more.

Working together, with Jessica’s supportive and knowledgeable leadership, we each made artefacts to be proud of. Over six weeks of Wednesday evenings in the MAKESHOP we learnt to sew conductive thread, programme micro-controllers and solder circuits, as well as make sense of the exciting electronic components we could combine with interesting fabrics in our designs

Finally we demonstrated our work in an exhibition in the Science Gallery on April 26th 2017:

Doireann’s glove instrument

Susan Reardon's jacket — Susan Reardon’s jacket

John Hegarty's bowler hat — John Hegarty’s bowler hat

Una O'Malley's scarf with loudspeakers — Una O’Malley’s scarf with loudspeakers

Katrina Enros' badge — Katrina Enros’ badge

Caroline Kelly wearing her necklace made with handmade felt, slices of stalactite and LEDs next to Richard Millwood wearing his LED lit bowtie, braces and beating 'LED by the heart' decoration — Caroline Kelly wearing her necklace made with handmade felt, slices of stalactite and LEDs next to Richard Millwood wearing his LED lit bowtie, braces and beating ‘LED by the heart’ decoration

I am now wondering how far this can go.

The initial premise was for me to find a course to fulfil my own creative aspiration. I now think that it may be a route to learning about programming and technology, starting with our desire to be crafty and creative, building from where we are already comfortable in making things, to add a desirable electronic aesthetic dimension. Having broken the ice with this encounter, perhaps participants will find a better relationship with programming and technology, or at least a greater clarity about how such things work.

So now I feel it may be the basis for an adult education model, and so intend to pursue this as an idea for Art teachers, working with the Art Teachers Association of Ireland and the National College of Art and Design and of course MAKESHOP!

I also think it may be interesting to explore the idea with a more general public, by seeking support from Enterprise Ireland to establish feasibility.

Personally I am now the proud owner of two Adafruit Flora Arduinos – small computers usually called micro-controllers, two BBC Microbit computers and lots of lovely LED swag – I can’t wait to make the next mad idea come to fruition!

Thanks to Doireann Wallace & Jessica Stanley for working with me, to all the participants for working so hard and to Nadine McDonogh Cunningham & Rozenn Dahyot for the photographs.

23 February 2011

Analysis of a single interaction

I revived this Analysis of a single interaction recently thanks to a PhD student who is working on user-centred design. It was first developed in 1988 based on Donald Norman’s work, applied to the kind of computer software we were designing then. The ‘concept keyboard’ mentioned was a programmable touch pad from that era which enjoyed considerable popularity as it allowed an interface based on the developer’s own visual layout, suitable for younger and special needs pupils – a precursor of the iPad!

17 June 200717 November 2007

Royal College of Art Show

Platform 8:

“The Chinese Government has recently commissioned the building of more than 1600 new design colleges, with a view to ending the division between the design of an object and its production.”

This was a very small part of an astonishing show in a large tent in Hyde Park, over the road from the Royal College of Art. The students on the MA Product Design had been organised into six ‘platforms’, focii led by their tutors. The words quoted above, from the Platform 8 poster they were giving away, are fascinating in the context of Stan Ower’s work around the culture of tools and gives me food for thought as I prepare for the next Owers Lecture.