08:30
How to Use Rayyan (Quick Start Guide) for Systematic Review
A tutorial walks through running a systematic literature review in the screening tool Rayyan on a free account, from start to finish. It covers importing a BibTeX file of references, detecting and resolving duplicates, adding inclusion keywords and exclusion reasons, title-and-abstract screening, labels, and full-text review with PDF attachments. Some conveniences — the duplicate auto-resolver and automatic PDF retrieval — are paid-only, and highlights around keywords can be toggled off.
Notes
How to Use Rayyan (Quick Start Guide) — Research Notes
Crash course on Learn Meta-Analysis (YouTube, presenter Noah), recorded/dated "as of today, May 8th, 2026." Demo runs on a free account throughout; paid-vs-free differences called out. Covers the full pipeline at "bare-bones basics" level only.
Create review + import
- Create review → title ("PA test"), type = systematic review, domain = "not applicable" (irrelevant).
- Import step: Rayyan now accepts direct PDF upload — "I think you can upload up to 10 PDFs at a time" (presenter calls this relatively new). Demo instead uses a BibTeX
.bibfile exported from a database, which is his usual systematic-review starting point ("I'm usually starting with database results"). .bib→ continue → files checked/imported automatically.
Duplicate detection
- "Detect Duplicates" button → proceed; runs in bottom-left. Detected 7 duplicates in the demo set.
- Auto-resolver is a paid feature (confirmed after trying it on the free account).
- Free account flow: side-by-side diff with differences highlighted; choose Keep Left / Keep Right / Keep Both, run through each pair. Presenter closed it with duplicates unresolved for the demo.
Review data page
- Shows review info: duplicate count, reference count; filters on the right.
- Keywords for include/exclude reasons are pre-populated by Rayyan — presenter deletes them ("I don't like anything being pre-put in there") and adds his own: include keyword "pedagogical agents" (the review topic), exclude keyword "appearance" (a term he knows recurs in excluded studies).
- "Screening Criteria" box: free-text inclusion/exclusion criteria, hit enter to add. Value: keeps criteria consistent when multiple people screen.
Title/abstract screening
- Records appear one by one in the center; keyword matches highlighted (exclusion = red, inclusion = green), highlight toggle on/off.
- Decisions: Include / Exclude / Maybe. Presenter never uses Maybe: "anything that's a maybe would be included for me." Also skips reasons at this stage because of volume.
- Labels: create a label (e.g. "review"), apply per study; labeled studies show it on screen and in the left list; labels become a right-side filter.
- Right-side filters sort/filter by included/excluded and stack multiple layers ("add another layer by sorting by filter"); left sidebar color-codes decisions green/red. Filtering is "one of the first things that ever drew me to Rayyan" — the reason he still uses it.
Full-text screening
- "Add articles" → data source = prior screening step → adds the included set.
- "Auto Retrieve PDF" button tries to fetch the full text from the web — failed for the demo article; manual attach instead. Uploaded PDF auto-matches to a citation (presenter brute-forced a deliberately wrong PDF).
- After you decide include/exclude you cannot return to neutral ("Well done, there are no articles left undecided"). Presenter advises opening the PDF first — it opens in a side panel while you decide.
- Reasons/labels matter here; custom exclusion reasons can be typed as free text (e.g. "undergraduate students" when the review is K-12 only), and new reasons appear in the right filter panel.
- Presenter's actual workflow: never uploads PDFs — keeps files in a desktop folder and uses Rayyan only for include/exclude, reasons/labels, and sorting.
Data extraction
- Presenter caveats up front: he uses his own extraction software, had barely tried Rayyan's.
- Flow: Add articles → full-text step → included. "+" button builds the form: column title, question type (demo: free text), required toggle, question group (demo: "blah").
- Click a study → opens in the PDF reader; answer questions; "Add answer" for notes; "Open PDF in new tab" for dual-monitor work; Save Changes → Mark as Completed.
- Export = CSV, delivered by email link; from the Rayyan library → exports → download zip containing
customizationandarticlesfiles;articlesholds the data.
"when you actually want to export that data, right now it is exporting it with a JSON schema in the actual CSV file"
- Extracted answers come wrapped in JSON inside the CSV column — needs spreadsheet functions (or an LLM) to unwrap; presenter calls it "a little complicated" as of May 8, 2026 and says he wouldn't use this tool for that reason. Copy-paste into LibreOffice Calc also failed/truncated ("dot dot dot"), which he advises against as a workaround.
Bottom line
- "for a free account, this thing is pretty stinking powerful." Presenter is enthusiastic about screening speed, labels/reasons, and multi-criteria filtering; this video shows only "the absolute bare-bones basics," not all of Rayyan's features.
Transcript · 20,601 chars
Hello, my friends. This is Noah and welcome back to the channel. Today, we are doing a super crash course on Rayyan. We're going to go through the full pipeline from start to finish, and this is not going to show you every single feature. This is just a crash course on the very, very basics. I do want to mention today we are using the free account on Rayyan. We are not using a paid account. I am logged in with a free account, so I'm going to show you what you can and can't do on a free account. So, let's go ahead and get started. I have created my account already, and what I'm going to do is go ahead and hit create review. We're going to add a title. Uh I'm just going to call it PA test, and I'm going to choose systematic review, and I'm just going to put not applicable for domain cuz it doesn't matter. Then, I'm going to upload my articles. Now, something I want to point out that's a relatively new feature of Rayyan is you can now upload PDFs. I think you can upload up to 10 PDFs at a time. I'm actually going to use BibTeX file because that's what I downloaded from a database, and when I'm doing a systematic review, I'm usually starting with database results. So, that's what I'm going to show you guys because that's what I typically have. So, uh I'm going to go ahead and I'm going to select that file and get it uploaded. Okay, so you can see I now have my .bib file. I'm going to go ahead and click continue. Goes here, checking my files, and it's going to get it all logged in. Perfect. Says we can invite members, but I'm going to work on this myself, so I'm just going to click skip. That brings us here. So, from this first page, the first thing that you might want to do is detect duplicates. You can press this detect duplicate button, and it's going to help you solve them. So, you click proceed here. It's going to do a little uh of its magic, and you can see it's running down in the bottom left, and we're just going to let it run for a moment. It really doesn't take very long, especially considering that your other option would be to do this like yourself or with some other type of software, which is going to require more steps. So, typically speaking, this works pretty quickly for me in the few times that I have used it. And so, I think in this data set I have like seven duplicates, if I remember right. Yeah, okay, we do have seven. So, it detected seven, and over here you can see it says unresolved. So, I'll just show you what this screen looks like. It brings up this thing where on the left and right you can see it highlights the differences between the two. So, they do have this auto resolver. Uh let's try that. Oh, nice. Okay, it is a paid feature. I think I was going to say I think this is a paid feature. I'm pretty sure it is. I'm going to try it anyway. Yeah, it is a paid feature to use the auto resolver. So, I'm not going to do that. What I'm going to do is show you just real quick what this would be if you are doing this on the free account. So, on the left side you can see what the differences are. They are highlighted compared to the right side. You can choose to keep left, keep right, or keep both. And so, you can just run through this for all of the articles that you have that are detected as duplicates. For the sake of time, I'm just going to leave them in and close this for now. So, the next thing I'm going to do is go to my review data page. This just shows us general information about the review, how many duplicates we have, how many references. And over here on the right-hand side, we have all of our filters. So, I'm actually going to delete these keywords for inclusion because And this is just my personal preference. I don't like anything being pre-put in there. I like to put them in myself. So, I'm going to go ahead and I'm going to delete those on uh the include and exclude reasons. Looks like it didn't take when I did the exclude reasons. There we go. Okay, so those are gone. And then up here, I'm going to actually add my own keywords that indicate what it means to me if I should be thinking about including the study. So, this is a subset of studies on pedagogical agents. So, I'm just going to type that in. And I'm going to click add. And then under here, under reasons for exclude, I'm going to put the word appearance for no other reason than I know that shows up in a lot of studies uh that are excluded for me. So, where this matters is we're going to go to Oh, there's one more thing I want to do on this page. So, up here, you see this where we have this screening criteria? I'm going to click that. You can set your inclusion and exclusion criteria up here. You can put them in. And so, this is really helpful if you have multiple people working on the review so that you always remember what they are and it's consistent among people. So, you can just type them in free text, hit enter, and it's going to show up. I'm not going to do that right now just cuz it takes 13 seconds and I'd rather move forward. So, I'm going to go to the screening step. This is where we're going to do our title and abstract screening. Over here on the left, we have all of our titles and abstracts. They come up one by one here in the middle. And you can see we have the word appearance here highlighted in red because that's what we put as a keyword for exclusion. Now, if you don't like those highlights, and personally I don't typically like highlights, I'm going to you can just push the button and look, it's gone. On, off, on, off, on, off. Kind of fun. And same thing for your inclusion reasons, they show up in green like down here with the file name. Now, the main things you're going to probably do in this title and abstract screening is either hit the include button, the exclude button, or maybe you're going to add reasons or labels. Now, personally, I typically don't do the maybe button. I only do include or exclude and anything that's a maybe would be included for me. That's just what my workflow has always been. In addition, for my title and abstract screening, I don't usually include reasons for that at this stage. I usually just do include or exclude cuz there's so many studies to go through. The other thing that I will do is I like to go through and add a label for every single study that is a review study. Okay? So, if I find a study that's a review, I'm going to create a label and I'm just going to type in review. I'm going to hit create. I'm going to apply it. And you can see it now shows up as review there on the screen and over here on the left side. The other thing is over here on the right under our filters, we can actually use these labels as a way to sort and filter over here on the right. Click it on, the only one that comes up is the one that I just labeled review. So, these filters and the ability to actually move back and forth between things is one of the first things that ever drew me to Rayyan. It's the reason I still use Rayyan because I really, really like the way that these filters work and the general workflow here. So, what I'm going to do is I'm going to go ahead and I'm just going to click a whole bunch, not a whole bunch, a couple include, exclude so you can see what it looks like. Include, exclude, exclude, exclude, exclude. Okay, cool. So, now you can see over here on the left-hand side, that shows the ones that I excluded and the one that I included with very obvious colors. Green for included, red for excluded. Up here, if we go to this filter, we can easily sort and filter by things that are excluded or included. And you can also add more layers. So, like if we wanted to go to I don't have more layers built in right now. Uh so, like if you have multiple labels and things like that, you can choose to only do included. And then over here on the right-hand side, add another layer by sorting by filter. So, okay. We're going to pretend that we have gone through and done all of these. So, what I'm going to do is I'm going to go to to full text screening. So, here it says, "Are you ready?" And I'm like, "Yeah, of course. That's why I pushed the button." So, I'm going to click add articles. It's going to bring this up and it says, "What is your data source?" And we have our screening that we already did. I had one included article. So, I'm going to click go ahead and add that one. What it's going to do is it's going to import this for me. Okay. So, here's a cool thing that I just learned about recently, okay? You see this auto retrieve PDF button? This works. So, like it will try and go out and find that relevant article for you. And in this case, you can see it wasn't able to find this exact one. And that's okay. Um so, what we can do if we need to manually attach this is we can click this attachment. So, I went ahead and I did that. But as you can see, we're now here and it says upload didn't start yet. But I have my PDF. You can select more if you need to. And I'm going to hit continue. It's going to upload that file for me. And it's going to try and auto match it to the correct citation if you have more than one. Now, that said, I am trying to match an article that is not the correct citation, okay? So, I'm just going to kind of brute force things here. And we're going to pretend that this is the correct PDF even though it isn't. So, now our full text PDF is here. And I'm going to close some of these little windows in the bottom left. And you can see down here at the bottom, we can now do our full text screening. Now again, I do not use the maybe button. I only use include or exclude, but something that is very important is that you include reasons or labels. And I mean, this is just for your own use for reasons or labels. I have always traditionally used labels and it I don't have a good reason for that. We can just use reasons, too. So, we need if something is excluded, we need to write down the reason why it is excluded. And if you work in a field like mine, you might see that none of these options here are actually particularly relevant for my field. So, what I can do is I can type in whatever I want. So, if I wanted to exclude this because it was say it was undergraduate students, and my study is only on K-12 students, you can just type that in and you can create it. So, I'm not going to do that right now because I need this to be able to get I need to accept this article so that I can show you the next step. But, as you start doing the exclusion reasons, uh they will show up over here on the right-hand side, and so you can do all of your filtering and stuff by that. So, we have very similar interface to we saw to the title and abstract screening. So, if we want to see the PDF, all we have to do is look at it, but it won't let me do it because I've already got this included. So, I'm going to click include again. Maybe I can Okay, I can't go back I can't go back to neutral. So, I should have shown you this beforehand cuz when I click full this, it says, "Well done, there are no articles left undecided." And when I click this, it's just going to it does end up opening it. So, that's a plus. Um All right, but normally what the workflow would be if you're doing this all in here is to go ahead and open up that full text before you make the decision so that it comes up on the side and you can make the include or exclude decision. As I said, you do not have to actually put your PDFs in here. You can just use this for sorting. So, what I've always done traditionally is I've always just had a folder on my desktop that had all of my files saved in it, and I never actually uploaded those PDFs to Rayyan, but I would use this still for include, exclude, and add my reasons. Um so, yeah, so there's many ways that you could choose to use this, but at the end of the day, the big thing is you have your include, your exclude, and your reasons and labels. So, that said, let's keep on chugging along. Uh they have this data extraction feature. Now, I'm going to say up front, I have not used this much. Um I don't know if this is new or if I just had never really noticed it, but I use my own version of data extraction software. So, I haven't really used this much other than in preparation to use this video. So, I'm going to preface this by saying this tool is pre- pretty cool and it works pretty well. But, when you ex- when you actually want to export that data, right now it is exporting it with a JSON schema in the actual CSV file. And so, if you are familiar with how to uh basically manipulate columns to maneuver some text and stuff out of a JSON schema, then this will be really easy for you. But, if you don't know how to do that, then you might not necessarily want to use this particular data extraction tool. But, at least in its current form as of whatever today is, May 8th, 2026. Um but, I'm going to show you how it works anyway. So, what we do, we go here and we click add articles. Like before, it's going to ask which step. Well, we want to do our full text screening step and we're going to say our included articles and we're going to click add. That is going to open up something like this. Okay, there we go. All I did was refresh the page and it came right up. So, here you can see we have our title. You can see it says we have our full text attached and it says press plus button to add questions. That's just over here on the right hand side. We can add in whatever types of questions we want. So, column title, I'm just going to say study and I'm going to say study ID and question type, I want this to be free text. I'm going to make this required. doesn't make you put a group. I'm just going to put like blah because I don't really care what the group is labeled. So, I'm going to say, "Sure, I'm going to create a question group and I'm going to call it blah." And I'm going to hit add question. And you can see what happens is now this gets beautifully updated and we have this really nice interface here um for working with this. So, what I can do is if we want to actually start screening studies, what we do is we can just go ahead and click on this. I clicked on the study. It's going to open it up in the PDF reader. It takes a little moment, but there we go. All right, so I have my PDF here that I can read and I can go through and I can answer all my questions. If you have more than one question, it shows up all in a nice little line here. And you can also add things if you need to just as like a note or something like that by clicking add answer. Uh the other thing that I want to point out is if you work on multiple monitors, you click this open PDF in new tab and that will let you put them on it the PDF on one monitor and retain this information on the other. So, I'm just going to type something in so you can see how it works. Peter and Adaso 2015. I'm going to click save changes so that it saves that into my coding form and I'm going to click mark as completed. And as I do that, you can see it put the information here. Now, this is the point I got I was at referencing earlier where you're probably going to need to get this information out of this system if you extract your data here, right? So, let's look at some options. First option is to click export. If we do that, it's going to export as a CSV and that's where I was saying I'm going to go ahead and I'm going to do that. I'm going to show you what this looks like. Um they send you an email with a link to go download it. So, I'll show you what that looks like in a minute. But the reason I was saying you may not want to use this tool if you are not familiar with how to get information out of a JSON schema is I found it a little bit complicated, at least as of today, you know, May 8th, 2026, a little complicated to get this data out in a way that's super usable to me. So, something I didn't try is I did not try just copy-pasting and that's what I'm going to try right now. So, I'm going to open Calc. Whoops, not my calculator. Not my calculator. LibreOffice Calc. And I'm just going to try pasting and see what happens. I honestly have no idea. Have not tried this. Okay, you don't want to do that. At least it's unformatted text. Let's try uh paste special, maybe text. No, that did nothing. Um we'll go back to uh back to here. And we'll copy it again, and we will just open up Calc one more time, and we'll try one more thing. Just straight Ctrl V, see if that work if that works or not. Um okay, this actually worked. Did it work? No. Okay, so here's the thing. You see how it says dot dot dot here on the screen, and then down here it says dot dot dot. So, it did copy it, but not all the text is visible. And so, it actually ended up copying that dot dot dot. So, let me expand this a little bit so that we can actually see everything. And now I'm going to try copy pasting again. So, I'm going to copy that. I'm going to go back to here, and I'm just going to do Ctrl V, hit enter. Okay, so now it does actually show it. I think that would be really tedious for me personally, and I wouldn't want to have to deal with that, trying to make all the text visible at the same time. I just wouldn't want to deal with that. So, I probably I would not advise doing this copy paste method or having that be your plan. Um let me go open up the CSV file that you emailed me so I can show you what the challenge is there. Okay, so here we are. Uh I'm actually in my library, and when you go into your Rayyan library, it'll give you your exports available here. So, I went through, I downloaded the zip, I opened it up, and you can see we have two things, customization as well as articles. We're going to focus on this articles because this is the one that is important to us and actually has our data. So, I'm going to open this up, and you can see it has our citation information. The one in particular that I chose didn't actually have any in that data set. That's a that's a BibTeX issue. That's not a export issue to my knowledge. And I'm going to scroll over, and here's what I was talking about. Let me zoom in a bit just to make sure you guys can see this really clearly, okay? So, over here, this is the data we extracted. We called the column study and the text we typed in was Schrader and Nadathur. So, as I mentioned, it's got a bit of JSON schema here, right? So, if you are familiar with Excel, you probably know how to remove this. Like, you can use functions and stuff within Excel or whatever spreadsheet software you use to remove this. But, if you don't know how to do that, that might seem pretty intimidating. I think an LLM would probably be able to help you. But, for me personally, as of today, that's not really a step I would want to deal with because I I simply I don't have to, right? There's other software I can use to do that. So, I think it's really cool that Rayyan has this data extraction software. I'm super excited that they are continuing to develop it. Um I'm going to I I'm going to give them some feedback about this JSON thing just because it's I know it's not something I would use because of this. But, that's why I wanted to warn you guys before I talked about the data extraction step because you might just not want to use it. You might want to use all the rest of the steps within Rayyan. And to be honest with you, I absolutely love Rayyan, right? Like, the ability to go through I'm trying to open this up here. My ability to go through and screen studies so quickly with include and exclude and add my reasons and labels, that's the thing that first brought me to Rayyan, being able to easily filter based on multiple criteria and get that information. And for a free account, this thing is pretty stinking powerful. So, I really like Rayyan. Hopefully, this gave you a quick overview on how to get your studies in, how to remove duplicates, how to do some really basic screening. I do want you to know, this is nowhere near all the features within Rayyan. Okay? Rayyan has all sorts of really cool features that are so much deeper than what I showed you here. I showed you the absolute bare-bones basics, okay, of how to get this up and working to try and just get you experimenting with this software without having to spend tons of time trying to figure it out on your own. So, that said, I'm going to stop talking here so I don't ramble on forever. I hope this has been helpful for you guys. Uh if you have any questions, feel free to leave them in the discussion down below. Um so that said, I hope you guys have an absolutely wonderful day and I will see you in the next video.