Multimodality & Knowledge Visualization
Two Options: Analysis or Making (Select One)
Option 1: Seek out and find a complex multimodal ‘site’ or ‘text’ (or a set of related sites/texts/objects) and do a thorough multimodal analysis of how the texts, objects, and/or sites ‘work’ to communicate ideas/meanings and/or practical knowledge (doing, action, skilled practice). You will need to use the methods, discourse, and analytical tools provided by the readings thus far to detail out the functions, strategies and nuances of the ‘text’ in making meaning. Consider key ideas from the readings and discussions, including: semiotic analysis (sign systems, modes, modal ensembles); frames, contexts, sites of display, genre, ‘transduction’ and ‘re-contextualization’ (see Kress; Bezemer & Kress); rhetorical and/or narrative strategies; communicative aims and purposes of the designer and experience of the user (you); points of multimodal interest as well as possible critiques of the text/making (evaluating the ‘success’ or challenges of multiomodal composition).
The key to this one is to curate a text to analyze that is sufficiently rich (complex) while narrow enough to focus on details. Example texts can be short videos or analysis of digital texts (see below for points of departure, or varieties of examples you might use).
Analyze SmART : http://smart-toolbox.eecs.yorku.ca
(website home page, video work, related tools/interface design, etc.)
For the Ted Talk, you might examine multiple levels of media and multimodal experience (the book, the TedTalk and the video of the talk).
Option 2: Multimodal Making:
‘Role-Play’: You are a ‘designer’.
First, draw upon your own interests, expertise and experience (in language learning, library science, film/cinema studies, the arts, design, making, etc) to share conceptual and/or practical knowledge multimodally.
Use tools like Photoshop, iMovie, a digital timeline, iBook creator, or some kind(s) of multimodal production tool to design a multimodal document that visualizes or models ‘knowledge’/’know how’ in a clear and specific way supports engagement and understanding and/or doing and applying.
By asking you to design a multimodal ‘text’, this means you may use (or combine) the media tools of your choice, including digital and non-digital tools (e.g., you can use watercolours, brush, and paper to model a technique that is then digitally captured and embedded in a multimodal document).
Models: In visualizing knowledge and/or know how, please focus on design/aesthetics, and relations between different ‘modes’ in creating meaning or modelling ideas/practices. Consider aspects of ‘narrative’, rhetorical strategies, or any other supports for understanding and reader uptake (see slide deck examples).
Objects: As the ‘object’ of your multimodal-ensemble, take something relatively complex but specific – a theoretical principle from physics, astronomy, literature, or the arts, etc; a historical/social event; a practical skill related to making in any domain or discipline. But keep things narrow enough so you are not overwhelmed (e.g, try to keep things within a one or two page document, or very economically crafted video work, etc)..
You can also consider this application as an opportunity to explore tools you might use for the final project (as a ‘warm up’ for you project).
Here is a list of tools, but feel free to seek out your own. Note, if you use ‘template’ based tools, consider if/how the template might constrain your ability to communicate.
Knightlab Digital Storytelling (Knowledge Visualization) Suite.
Pages integrates texts, images and graphics (knowledge visualization and modelling), video works, audio soundfiles, interactive min-presentations, 3-D modelling tools. Mac only.
BookCreator Much less powerful than Apple tools but works for PC and Macs.
iMovie or Windows Movie Maker (PC): Create video works, documentary films, visual essays, cinematic works (any genre), poetronica, experimental films. For newer PCs/Windows, you can edit/creative video films using Microsoft Photo.
Video Screen Capture: Free Alternatives to Camtasia (screen capture, tutorial production, editing digital stories, etc).
You can also use Piktochart and other “infographic makers” (google it), but you need to ensure that the templates do not get in the way of your ability to design and communicate (multimodally).