By Max Lovell

Note: This is an abridged version of an article that can be found on my personal webspace – see that version for more details, and updates.

Over the pandemic, running cognitive tasks online has become increasingly necessary. This post is a guide and collection of resources on programming languages, experiment hosting sites, and their integration with Qualtrics. If you have any feedback or additions to make you can email me at m.lovell@sussex.ac.uk.

Programming Languages

I’m aware of two main options: Python and Javascript (I won’t go into MATLAB here). JavaScript is the language most browsers use to do many of the more interesting things in websites, and sits between the more standard HTML. Python is an easy entry point for coding but isn’t supported by browsers, so if you want to upload an experiment online you must use the visual/graphical Builder view of the PsychoPy program so that it can be more easily translated into JavaScript with the help of their PsychoJS library and uploaded to their hosting service Pavlovia. Note that the builder view of PsychoPy is somewhat limited, and didn’t have the capability to create the task I needed.

So, if you want the ability to create any experiment you like, and run it for free online, you’ll need to go with JavaScript. The added benefit of learning this is that this functions as a more complete introduction to not just coding but web development as well. Similar to PsychoPy/JS, there is a free library of JavaScript tools for creating psychological experiments and surveys called jsPsych. The jsPsych website has great introductory tutorials for both JavaScript and jsPsych.

The only thing some of the tutorials above are missing is that you need an ‘Integrated Development Environment’ to type your code into, such as Sublime.

Experiment Hosting Sites

Once your study is up and running on your own computer, the easiest option to get it up online is using an experiment hosting site – all of which can be integrated with a Qualtrics survey if you need to use that to manage participants in a longitudinal study. Cognition.run is free and easy to use – just create a new task and upload your javascript to the source code section! It doesn’t accept HTML, so your CSS will need to be in a separate file. Pavlovia is run by the University of Nottingham staff who created PsychoPy and can be integrated with jsPsych – and only costs 20p/participant. Gorilla can also accept jsPsych code but is more costly – if you qualify for the researcher package you could get your first 700 participants for £500. Note that Gorilla use the previous version of JavaScript (ES5) and any code in ES6 won’t work (they have told me this should be fixed by the end of the year). Other hosting options are listed here.

Hosting sites in Qualtrics

Qualtrics is incredibly useful to keep track of participants longitudinally by linking surveys. Experiments hosted on the above sites can be integrated into Qualtrics in a few ways.  The easiest of these is with an HTML iFrame: create a ‘text/graphic’ question and add the following code into the ‘HTML view’ <iframe height=500 width=600 src=“TASK URL”> with the right width and height (this can look a little janky). Similarly, this code will open the experiment in a new tab <a href=“TASK URL” target="_blank">Click Here</a>, although I’ve only been able to get this to work with Gorilla.

Finally, you could split your Qualtrics survey in two, redirect participants from the first one to your hosted survey, and then back to the second survey. Redirecting can also be done by adding JavaScript to your survey as follows: Qualtrics.SurveyEngine.addOnReady(function(){setTimeout(function () {window.location.href = “TASK URL“;},5000)});.

To keep track of participants between surveys: 1) set a random participant ID 2) embed this into the URL we send from the first Qualtrics survey 3) use jsPsych to grab that data from the URL 4) set up a redirect link by using on_finish(window.location.href = "Survey pt2 URL”) at the end of your jsPSych script in cognition.run, or Gorilla’s redirect node (see also), or the information here for Pavlovia (personally untested) and 4) appended it to the redirect link from the hosting site in a way Qualtrics will understand and record that data.

Direct Qualtrics Integration

It’s also possible to integrate your task into a Qualtrics survey question using their JavaScript question editor without using a separate hosting site at all. A few things to note: saving your data with JSON and not CSV is recommended, template literals will not work properly in Qualtrics, and Qualtrics relies on the JavaScript version in a users’ browser, so try to avoid using the latest version (ES6) to ensure maximum usability. However, with this approach your data won’t save to Qualtrics – you’ll have to use JavaScript to send data to your personal server space at the University. Unfortunately, the sections on saving your data won’t work if you’re at Sussex – I detail below how to use the PHP and web server method. There may also be a way to get MySQL set up on the university servers, or at Edinburgh University, or to use a Box app instead of the DropBox method listed here.

Webservers & PHP

Before you can follow along with the kywch.github PHP tutorial, you will need to set up a personal webspace with the university, and authorise your computer to access your N drive. I would recommend using the FileZilla STFP (software for transferring files). The university host is sftp://unix.sussex.ac.uk and the port is 22. You will also need to log in through your command line terminal with the command ssh[username]@unix.sussex.ac.uk and your password.

Now follow the steps in the PHP tutorial which make the exp_data directory within public_html. However, the commands  $ touch index.html , $ echo "DirectoryIndex index.html" >> .htaccess, and later $ chmod 772 hello-world won’t work on the university servers. Instead, use my changes to the PHP file here, paste it into your IDE, save it, and move it to exp_data using FileZilla – making sure to change the `/hello-world/hello_` to something more relevant to your own study.

Now create the data upload folder (e.g. ‘hello-world’) outside of public_html, using the same technique in the tutorial above. The command cd .. moves up a directory level, ls lists the contents of the current directory, and use pwd to check you are in /its/home/[username]. Create the directory where you want to save your data (i.e. ‘hello-world’) here. Check the Kwych tutorial above on permissions, but assign the following permissions instead: Hello-World = 1703, exp_data = 1701, save_data.php = 1704, public_html = 1701.

Next, create a file in your IDE called .htaccess with the following content:

    Options -Indexes

    Options -ExecCGI

    AddHandler cgi-script .php .php3 .php4 .phtml .pl .py .jsp .asp .htm .shtml .sh .cgi

    <Files ^(*.json)>

        order deny,allow

        deny from all

    </Files>

and move it to your uploads folder (e.g. hello-world) using FileZilla (you will need to reveal hidden files to do so, e.g.  .+⇧SHIFT+⌘CMD on Mac).

Now, follow the rest of the tutorial regarding saving data with JavaScript. However, delete the lines declaring the task_name, data_dir, and file_name variables – we just need sbj_id and save_url, and your jQuery.ajax() function should look like this:

    function save_data_json() {

    jQuery.ajax({

        method: 'POST',

        dataType: 'json',

        cache: false,

        url: save_url,

        data: {

            file_name: sbj_id + '.json',

            exp_data: jsPsych.data.get().json()

        }

    });

}

If you’ve followed everything else correctly, running your Qualtrics survey should save the task data to your target upload directory (e.g. ‘hello-world’) – you will need to refresh FileZilla before you can see the file appear. To download the data, as it can’t be accessed through a public URL, go to the left-hand panel of FileZilla called ‘local site’, which lists the contents of your own computer, and create a data folder somewhere that suits you. When you then download on the remote site on the right-hand side of the screen, the files will be downloaded to this folder onto your computer.

If you have any issues or feedback, email me at m.lovell@sussex.ac.uk

Max Lovell is a PhD student under the supervision of Prof Zoltan Dienes, Dr Clara Strauss and Dr Sarah Garfinkel. His research focuses on mindfulness as a form of metacognitive training, alongside general theoretical and experimental issues in studying mindfulness.

Find out more about our research on Cognitive Psychology.