Cloudy with a chance of Umbraco Awesomeness

VideoDays, Webinars, WebinarSpots, RawMode, Thomas Madsen-Mygdal, Steffen Fagerström, Webinars5 Okay, good morning everyone. It's quite bright up here. It's nice to see so many wide awake, happy faces. Thanks for turning up. I don't know if anyone noticed yesterday there was a TBA for this session. So you're probably thinking, oh, it's going to be something really exciting and, you know, kind of cool and surprising. So it's me. It's a bit like, you know, Glastonbury Festival. Everyone knows Glastonbury, right? And you have the Sunday afternoon and there's a slot and it says, this is a surprise guest. And everyone's going, oh, you know, it's maybe Rolling Stones. Maybe it's, you know, kind of someone like... Jay-Z or maybe Jimi Hendrix has come back from the dead and he's going to play at Glastonbury. And then it turns out to be like Kanye West and you're a bit disappointed and, you know... But you enjoy it anyway because it's Glastonbury. Similar. Similar thing is going to happen today. I'm Matt Frigby. This is Cloudy with a Chance of Umbraco Awesomeness. Thank you. So a little bit about me. I'm a freelance umbraco. Braco and .NET developer based in Bristol. Any Bristolians in the audience? Hey, excellent. Three, four. Brilliant. I've been working with Umbraco for about four years and spent about 15 plus years in IT. Doing various bits of kind of IT support, operation management, that kind of stuff. Been a developer, proper developer for about the past five or six years. So, yeah, I've got a blog. Doesn't everyone? And I'm on Twitter. And that's me before I worked with Umbraco. So a lot less grey hair, smiley face. Still smiling. That's good. So the project that I want to talk about today is a project I've been working on for about 14 months now. 14, 15 months. And it's a company called Nessie Learning. And Nessie is a Bristol-based company. And they specialize in... Online training for children with dyslexia. So dyslexia, if everyone's aware of it, is kind of the... Or the kind of... Well, seen as a disability to understand written language and take it in and kind of repeat it in the right way. So the information somehow gets kind of muddled up inside the brain. And what you think is the right way to spell a word is the wrong way to spell a word and that kind of thing. And there's lots of different types of dyslexia. And there's lots of different... Teaching techniques to help children with dyslexia learn how to read and write. And numbers, do numbers and stuff like that as well. So Nessie has been going for about 10 years. Mike Jones, the guy who founded Nessie, is dyslexic. And he went through this whole learning process himself. Brought all these learnings that he had into the software and into the online learning platform. They started out doing interactive CD-ROMs. So... Proper old school, you know. Put it in and you get the auto run and the shockwave movies. And all that kind of stuff. And now they've been online for about five, six years. And they've got massive presence in the UK and a big presence in the US. And they're building their US presence as well. And then the plan is to go further afield and go into different languages and different countries and that kind of thing. So they target... Mainly target schools and education systems. People can buy single packages. So if you have a child who's dyslexic, it's well worth checking out Nessie.com. I don't get any commission on that. And then if you're a teacher... No, no one's teachers. A lot of schools use it to help children with dyslexia. So... The project that I came in to do... So there's me and another guy. Who are both .NET developers. Who are doing this project. The Nessie platform was a series of Flash games powered by XML config files. So loads of hard-coded, massive, massive XML files that powered a series of games. And then all that data was fed into a back-end database. And all the reporting and number crunching went on in that database. So obviously the drawbacks of that with... With iPads and iPhones not supporting Flash was a big stumbling block for them. So about a year and a half ago they kicked off this project. They brought in an agency. Another agency from Bristol to transform the Flash games into HTML5 games. So they would run across all devices. And then they brought in me and my colleague to bring the back-end, bring the API up to standard. So... Our brief essentially was to serve configuration data to the games. So the HTML5 games would sit there on a website. Call our API. Grab a load of configuration data. And then you'd play the games. We'd capture the results in the back-end. And then we'd send them all to the reporting system. So caption store data sent back from the games. They also wanted to be... Because they're global. Because they've got UK and US presence. With a view to look at. Looking elsewhere as well. They wanted to deliver all the media for the games through some kind of CDN. So that was a big, big requirement. Because they had a lot of video, image, sound. Everything that needed to be as close as possible to the end users. Data in the platform needs to be localizable. So even with UK and US. This is all about language and spelling. So it's very important that you've got the right spellings. And also... In the audio files. They have the UK ones narrated by a UK accent. And the US ones narrated in a US accent. Because the children need to be able to understand and relate to this. So it's very important. And then finally they needed something that was scalable. Because they have big plans to state this platform global. So why did we choose Umbraco? It's kind of lucky that we did. Or else I wouldn't be here. Or working on the project. So... Well actually... We started off the project with no Umbraco in mind whatsoever. We went in purely to build this API. And we quite quickly realized that Umbraco was a really good way to do it. So I'll get to that in a second. But again, some more requirements that we had to think about with the data. We had configurable data versus static data. So configurable data would be, say, a word in a game. So how is that word spelt? And maybe you type that in and you've made a mistake. So you can go into the back end and fix that. Or you want to change out an audio file to a different audio file. So all this stuff that... As someone managing the configuration, you need to be able to switch in, switch out, and update stuff. Static data. Things like language. Location. All these kind of things that are purely... Static. There's lots of bits of configuration data that drive the games. That if you change that, if you spell it slightly different, if you use the wrong case on it, then the games will break as well. So we had static data in there as well as configurable data. So in the Flash version with the XML files, they had something like 15,000 lines of data across all the XML files. So a huge, huge amount of data. So to manage and maintain. And if you think Mike, the guy who runs this, is dyslexic. And he's not technical. He's a designer. So he doesn't really want to even go into that. So he would go through these... Hunting through these XML files. Trying to find one word that was misspelled and then change it and then upload it. It was just... It was crazy. They had like 60,000 pieces of property. Like 60,000 properties. Property items of data. So the word has like a word, all the different letters in that word. The order of the letters in the word and all this kind of thing. So lots of configuration data to manage. Again, in XML files, which was just crazy. Localization. So they wanted to be able to take one version of the platform and move it to a different country, move it to a different language. So we needed to take that into account. And then... And finally, they wanted something that was visual and simple to use. Hunting through those XML files is and still will be painful if you've got that much data. Currently, we've got the service live in the US and the UK. And between both of those countries, we've got about 23,000 content items in Umbraco. And if you think... Probably average about five property items per those content items. You've got like 200. You've got like 220,000 bits of configuration data. So this is... There's a massive amount of data to... If you're doing it the old school way through the XML files, you'd just be there. Well, you'd probably quit or something like that. I probably would. So some design considerations. And you'll notice as I go through these, the slide curves, that's the design consideration I put in the slide. It's quite nice. Totally by accident. So we have this static versus editable, as I said. Some things have to be static. And if they change, then they break the site. Other things need to be editable. So if you want to switch a word out in a game, put a new word in. Or if you want to upload a different media file. So if you've got a better recording of that word or that phrase or something like that. You need to be able to switch this in and out. We have this... We have... We have this concept of configuration versus gameplay data. So the configuration is what we push out the API. So everything that the games are using to run. And then the gameplay data is kind of the progress data that is pushed into the API from the games. We had to think... See, it's nicely slanting. We had to think about referential integrity between our Umbraco data source... And our custom data source. How could we manage the relationships between node IDs and IDs in our custom data? So I'll show you how it all hangs together in a second. But in the early days, that was a real challenge how to do that. And we tried various ways. And the way we came up with eventually... I think... I mean, it seems... It's been running for eight months, something like that. So it seems to be going well. But... Platform stability. So we needed something that just was there. And we just rely on. And we don't have to worry about whether it's going down. Or whether the file is corrupt or anything like that. So we just needed something solid. And then we also needed something that we could deploy to new databases, new environments, quite quickly and quite easily. Ultimately, we went with Azure in terms of hosting. And a lot of this talk will go through. All the different bits of Azure that we're using. I think in terms of stability and in terms of deployability, for me, Azure is just fantastic in those areas. And we're using lots of different Azure kind of services as well as part of the project. So the Umbraco setup we used, it's probably the most basic Umbraco setup you can think of, really. So it was version 7.2.3 when we started. And now we've got the latest 7.4.3. We're using the Azure blob storage provider. So that's a package that Dirk Seafeld, is he in the room? No? Okay. Well, thank you, Dirk, anyway. We're using that. So because we're hosting everything on Azure, it made sense to just stick all our content into Azure straight away and not worry about it. We're using USync. So Kevin, jump. No? Everyone knows USync. So doc types, data types, templates, everything that we do in Umbraco, use USync, deploy it to the next environment, and everything is just there. In terms of templates, we have two Umbraco templates. And I'll show you them later on. I'm very proud of them. We're using SQL Azure 2012. And everything is hosted on Azure. So I'll just give you a quick view of the back office. I'll just show you how. There we go. That's big. Okay. Okay. Oh, hang on. I need my glasses for this. There we go. Right. So first of all, document types. So we've got a lot of document types. And it's kind of, we designed it this way because we wanted to make, so you can see, a huge amount of document types there. We wanted to make it as simple as possible for the people. Managing the content, managing the configuration data. We wanted to guide them through the steps of how you put this thing together. And it actually worked really well. So when we initially set up the content tree, then I showed Mike, you know, this is how you do it. This is, you know, click this, add new item, all this kind of stuff. And then he brought three, like, school trainees in. So kind of 16-year-old, you know, kind of people looking for work experience. He brought them in. Sat them down at PCs and just said, this is, you know, this is all the data we need to input. Off you go. And they just went through and did it. And it just worked. There was no, well, hardly any questions from them to me. Mike could deal with all the questions. So it just worked really well. And part of that is because we, it was painful to set up in the back end because we just had to create every single item as we went along. But it meant that they were guided through the process of setting up the content with virtually no. No room for error. Everything was just so specific. So, yeah, loads of content items. There's hardly any, where we are, developer. In terms of the data types that we used, everything is pretty much out of the box. There's nothing in there that's special. We've got, like, multi-node tree pickers. And we've got drop downs and media pickers. Where's media? Media picker. I can't find any media pickers. Anyway, that's literally it. Everything is either multi-node tree picker or drop down. And we get to work with cool stuff like this. What properties do you want? Do you want a baguette, banana, bricks, bone, cactus, carrot, cheese, chili? So all these, because it's all geared towards kids. All of the games are based around, like, animals. And gorillas and monkeys and bananas. And it's just, it's brilliant. It's just really fun to work with. And all the animations are just so funny. And all the audio sounds, all the monkey noises and all this kind of stuff, it's Mike, the guy who runs the company. He does all these sounds. And he goes into a cupboard with his iPhone, puts a, like, blanket over his head, and does all the sounds into his iPhone and then puts them into the, it's just so cool. Like, really low tech, but just works so well. So, yeah. So, yeah. So, yeah, there's nothing special about this. We did start to think about maybe bringing ArchType and, you know, build these kind of complex properties so that we could then populate and make it dynamic. But then we just thought, that's just too far. We don't need to do that. We'll keep it as simple as possible. And then we're reducing the complexity, reducing room for error. So, we have UK and US sites. Yes. So, we built the UK first and then literally made a clone of it and then just someone went through and changed all the different language pieces. Within here, we have a number of applications. So, this is the main application. This is the reading and spelling is the main application that we built. It's around, we just put another one live, this dyslexia quest. And this, once we have this kind of framework in place, the data set up here and the API, literally, it took like a day to get this data in, get the structure, get the API, push all the data out. The games on top of that took a little bit longer, but, you know, we had an agency doing that. They built the games on top of dummy data. We built the API. It's like the channel tunnel. And then we saw almost met in the middle. So, you literally, you can't really do anything else in this application apart from, you know, create a path, create a folder, and it guides you through as you go along. Within this level, you can only create a game. So, every single piece of data is just guiding you through that creation process. If we look at the actual domain application, so, the reading spelling application, we've got a set of skills. So, these are all the different skills that dyslexics kind of require or, you know, have weaknesses, strengths in. And again, there's nothing there. There's nothing special about these. They're just basic text boxes. You have a set of missions. So, when you log in, it says, you know, go and do this island, go and do this, go and take the challenge, go and, you know, test your reading skills. Then the games are basically a set of ten islands. And within those ten islands, you have ten lessons. And then within the lessons, you have what we call, learning objectives. And each learning objective then has a series of games or videos or worksheets within those. And again, the content is such that, at this level, you can only create a lesson. You know, it's kind of standard stuff. And this is how you kind of build your own bracket websites anyway. But it just meant that, like I say, they could sit a bunch of, you know, 16-year-old school trainees down with a sheet of all the stuff. And they just went through and entered it. You've got every single game has got a different configuration. So, slightly different words, slightly different kind of, this one's got an image here, this one's got a sound. You know, so, we, in the back end, we did a lot of duplication of properties. So, each document type, we were repeating properties a lot in those. But we thought it was worth it to keep the front end as simple and as intuitive as possible. And that's kind of it really. In terms of a structure, it's very, very simple. There's no kind of special magic in there. But it just worked and it just got the data in. So, if we go back to the slides. We have a, when we inherited this site, like I said, we had a .net site that was serving Flash games using XML config. We had and we still have a WordPress site that does all the commerce, which is a bit disappointing. Using MySQL and using WooCommerce. So, I've had to do a little bit of PHP, which just feels wrong. But not too much and I kind of, you know, I've got over it. So, that's fine. And that all pushes customer data into Salesforce. So, luckily, I don't have to touch that, which is good. We have the main application. So, the main application is where people go to play the games. We've got the API that we've built on top of that. And my cursor has now disappeared. And then everything goes into SQL Server. We've got a reporting section, which is kind of a vanilla MVC. Well, MVC with Angular in there. And then we have the CDN, which we're using Azure CDN. We've got the CMS, which is Umbraco. Hey. And then we've got a customer service portal, which is kind of for their back-end support. So, quite a lot of pieces to go into the system. And then this is how the Azure bit looks. So, this is how all our .NET bit is holding together. So, where do we start? So, we've got Umbraco at the top. And we've got the Umbraco database right up there. That's just disappearing off the top. We've got another database here. So, we kept the two separate. This is our custom data that drives the API. When we first started on the project, we put all this into one database. And we just had all the Umbraco tables, all the Nessie tables. And that seemed to work fine. But then we had real problems deploying content between different environments. And we were working on the API structure at the same time that we were working on the Umbraco structure. And it just didn't quite fit together. And we were trying to work out backup plans. How can we backup this data? How can we have the true source of data? And again, by having them both in the same database was just causing lots of problems. So, at the point we decided to split them, we initially had foreign keys between the node IDs and our custom tables. Which seemed to work well. Because in our data, in our custom data, we need to know which node is responsible for that data. So, if we then need to change it, we can go back into Umbraco, change it, and there we go. So, at first they were foreign keys. Which kind of worked well. But then we thought, well, what if that's deleted from Umbraco? Because you want to get rid of that game and put another game in. So, then that foreign key would have to be nullable. And then that just felt a bit wrong. So, in the end, we just lost the foreign key relationship anyway. So, we kind of risked a little bit of data integrity there. But it seems to have worked so far. And now if we delete stuff from Umbraco, we can just remove the foreign key from our custom database. And then that item is just effectively deleted. So, the two databases are split. With Umbraco, we've got the media storage, which is using the Azure blob storage. We've got the API, which is using, which is hosted on a cloud service, Azure cloud service. The website is on the cloud service. What have we got? Oh, we've got a client there. They've got a web browser. CDN is delivering all the media. So, Azure CDN is delivering all the media. And then over this side, we have Azure message queue. So, we're using Azure queue storage to manage our message queues. We've got a thumbnail generator for PDF files. So, I'll go through that in a little bit of detail in a bit. And then we've got a weekly reports process, which uses, quite a few bits of Azure, which I'll go through in a second as well. So, the API is Web API 2. We're using Entity Framework. We're using Image Processor. And we're using Umbraco, which is cool. And then cloud services, SQL, blob storage, message queues. Oh, there's also Task Scheduler, which runs a weekly report. We've got Send Grid at the bottom. Does anyone use Send Grid? We've got Email Delivering. So, it's part of the Azure offering. So, you can get it from the marketplace in Azure. Really easy to sign up. I think you even get something like 50,000 free emails a week or something like that. And we're nowhere near that limit. So, it just works perfectly. The only time we approach that limit is when the system starts sending me email alerts that something's gone wrong. And then we have to get on it quite quickly. So, lots of Azure bits in there. For the reporting site, which is... Effectively, we've got two different servers for the website and the reporting, but we've got them in one there. So, we're using just a straightforward MVC site with lots of Angular bits dropped in. Again, taking Entity Framework to go to our custom data and not the Umbraco data. And the beauty about this setup is that we've got dev staging and live, as you do. And then we work in dev and our Umbraco database is in dev. And then we've got our Nessie database in dev. If we want to move everything to staging, rather than recreate all our Umbraco data in staging, we can just point the connection string for our dev database to staging, publish everything, and it goes to the staging database because of the way that we're intercepting and doing our custom publish of the events. So, that's quite nice. For production, we will recreate the data structure in Umbraco and then publish it to production. So, we need to make sure that that data is perfect in there. But it's quite nice just to be able to get something up and quick, any data up and quick into whatever environment you want to do. So, to do all this, we're essentially intercepting all the Umbraco events and then doing something with it in our custom database. So, we're intercepting a content service saved event. Particularly, well, only for the language. So, we have a language drop-down in the data types. And when we save that language drop-down, that needs to go into a database table in our custom database because everything hangs off that language. So, the application level, right at the very top, has a language. And if that language drop-down from Umbraco isn't saved, then we can't do anything with the application. The main one that we intercept is the published event. So, every single time you publish something in Umbraco, we'll capture that. We'll see what doctyping is. We'll see what type you're publishing. And then we'll do some custom transformation of that data, stick it into our custom database. We capture the content service trashing event. So, when you move something to the recycle bin, we'll capture that. And when you do that, we add a date deleted into our custom database. So, we'll keep the node ID in there in case you deleted it by accident and want to bring it back. But we add a date deleted. So, this item shouldn't be shown to the front end, shouldn't be shown to the games because it's been deleted from Umbraco. And then also, we'll do the deleted event as well. So, when you actually empty the recycle bin in Umbraco, it will go through every single piece of content in there and remove the node ID from our custom data. So, there's no relationship between our custom data and the now deleted piece of Umbraco data. And finally, when we upload media, as I mentioned, we've got this thumbnail generator working on the PDF files. So, when we upload a PDF file, we create a thumbnail from that. So, we're intercepting the saved event on the media service to be able to do that. So, I'll go through those in a bit of more detail. So, publishing Umbraco content. You publish the Umbraco content item, and then you intercept the unpublished event. Then we're creating our custom entity, or we're updating it if it already exists. And then we continue. We save that to our database, and then Umbraco continues with the publish process. So, is everyone familiar with intercepting events? There's a few. Yeah, there's a few hands. If you're not, there's a really catchy URL across the bottom. Intercepting Umbraco events. So, I've written a couple of blog posts on this, if you want to go through it in more detail. So, essentially, we're overriding our Umbraco application event handler. And then what we do, we set some web config flags, true, false, so we can switch this on or off quite easily. So, if we don't want to push everything to our custom database, we can just set them to false. And this whole event triggering, event interception and custom data manipulation doesn't happen. So, that was very useful when we were trying to get the two structures of the API data and the Umbraco data just right, because we were causing lots of problems backwards and forwards. So, essentially, we look in the web config, and we get the flag. And then if the flag is true, so if we do want to save the content to Nessie, then we just call this literally one line, which calls our custom publish service. So, here's a... If everyone saw the anti-patterns talk yesterday, this might be a candidate for that. We'll see. Is Mark in the room? Good. He's not here. Don't tell him. So, the only way I could figure out how to get this to work was do this massive switch statement on the doc type alias. I don't know if there's any other better way. But essentially, this is intercepting the published event now. And what we're saying is, for each published node in the published nodes, so whatever we're publishing, whether it's a single node or the entire tree, then we go through this big switch statement. And let me show you how big it is. It's probably my finest switch statement ever. I'm just going to make it really big just to make it more impressive. So... There we go. Thank you. Thank you. I don't know any other way to do it because we've got all these custom doc types. So, if there's a better way, if anyone knows a better way, I'm not going to rewrite it. But it would be useful for the next time. Yeah. Each one fires off a specific function because each one is a custom set of data. So we need to get it into our own custom entity. Then put it into our own custom database table. So, yeah. Don't tell Mark. That's that bit. And then what we do then, we just... We're literally... First of all, we get in the language. So, like I said, that hard piece of data in Umbreco, whether it's US or UK at the minute, we get that language ID from the database. And then we're using entity framework to check if there's already an existing entity in our database. If there is, we'll update it. If there isn't, we'll create a new one. So, again, this is just normal kind of Umbreco, you know, content.getValue type stuff. So, nothing kind of crazy in there. Occasionally, we have to do things like... Where is it? So, I think, yeah. So, the getLanguageID, because we're using dropdowns, we have to do the whole horrible kind of Xpath node iterator thing. So, to get the value of the dropdown, we have to go into the data types, find the data type with our name on it, and then go through that, find the actual selected item, bring it back, get the ID. It's not pretty, but it's literally, you know, write it once, forget about it. And then, pretty much the same thing with the trashing event. So, what we're doing with the trashing event, when it's triggered, so it's triggered in the same way. I've duplicated the entire switch statement for the trashing event. So, that's two. And then, exactly the same thing. So, we get it from the database, and if it's not in there, we just return anyway, because we don't need to return any kind of, you know, success or failure kind of thing. So, we return that anyway. If it is in there, we set the inactive date to null, and then we save it. So, again, nothing complex. I quite like simple code. Deleting data. So, this is when we actually remove stuff from the recycle bin. We just set the node ID to null in our custom entity, and then save that back to the database. So, we're not deleting data from our custom database. We're just deleting the reference to any Umbraco data, because then if we recreated that same piece of data in Umbraco, we're not going to get duplicates, because we'd have one row that's ignored, because it's deleted, and it doesn't have a node ID. The next row would not be deleted, and it would have a node ID. So, our API would pick up the row that was new, that was active. So, saving Umbraco media. So, slightly different process, because we have to check the whole pipeline, and the core point of this is we have to check if there's a PDF that's being uploaded, and if it is, we have to do something special with it. So, the Umbraco media item is saved. We intercept that saving event, and then we create the custom entity like we did in the previous one. So, we'll check if it's in our database. If it isn't, we'll put it in. If it is already in the database, we'll update it. And then we do a blue thing, which is, is it a PDF? If it is, then we do something different. If it isn't, we just continue with the normal Umbraco save process. So, if it is a PDF, we get the node ID, and we get the file path from the item that we're saving, and we put it into an Azure message queue. I'll show you the code for that in a little bit. And then we... The Azure message queue then triggers a process called a web job. If anyone's used Azure web jobs, they're just amazing. If you haven't used them, then you need to check them out. They're just awesome. And I've done a couple of blog posts on them as well, but the URL was a little bit unfriendly, so I didn't put it on the slides. So, we generate a thumbnail of the PDF, and then we save that thumbnail to our Azure storage in the same media folder. And then we add the URL of the thumbnail into our custom database. So, our custom database now will have the PDF title. It will have the link to the PDF file. It will have the link to the thumbnail URL. And then it continues with the normal Umbraco save event. So, there is one slight hack that we had to do. So, on Azure cloud services, the local storage is something... It's like 800 meg or something like that. Is it 800 meg? No, 800... Yeah, that's about right. 800 meg, I think. And it was filling up really quickly, because we're uploading videos, we're uploading images, MP3s, and that whole thing was just... For some reason, it would just... It would fill up, and it wasn't clearing down. I don't know if there's any way that it should clear down, or we've done something slightly wrong to it. You know, we defended it in some way, and it just didn't want to clear down. But it just wasn't clearing down. And every now and again, the whole server would just die because it ran out of space. So, I'd have to go remote desktop onto the Umbraco until then. clear it down, and get it up and running again. So, in the end, I put a little hack into this whole process. So, when we've processed the PDF file, and we've saved it to the database, they just put a few lines of code in there to say, now go to this folder, and just delete anything older than, like, two hours or something like that, and just keep it at a real minimum. I don't know why that happened. I don't know if anyone else has had similar problems. And please let me know. But yeah, I checked on the hour forums and asked the questions, and no one seemed to know why it was happening. So, and that, it works. It's a hack that works, which is fine. Maybe. Depends what kind of school of development you're from. So, adding data to the message queue. So, Azure message queues are just amazing. You can put anything into them in, you know, kind of in a string format. You can put serialized JSON in there. You can just put, you know, a value in there. You can just put random text in there. And then you can use that message queue to trigger other things. So, like I said, when we save a PDF, we get the file path of the PDF, and we get the node ID of the PDF. And then the next few lines, essentially, we're creating a message queue data item, which is what Azure stores in the message queue. We grab our storage account. We create a client. So, you have to create the queue client to actually access the queue in there. And then we actually find our specific queue. So, you can have multiple queues set up in Azure storage. Has anyone used message queues in Azure? One, two, three, four. Yeah, okay. About five or six people. So, it's worth, if you're using Azure for hosting or for development, it's worth checking them out. They're really, really good and, like, really, really simple to use. The one gotcha in the message queues is, see the line where it says, where has it gone? So, the message queue name, so that's stored in our app settings, but that message queue name all has to be lowercase when you're setting up your message queue. If you use camel case or anything like that, it just won't work. I don't know why, but it all has to be lowercase, and that took me a while to figure that out. So, you get your message queue, create it if it doesn't exist. All this is just standard Azure storage kind of SDK stuff, and then we serialize our object into JSON, and we just stick it into the message queue, which is brilliant. What happens then is that we have what we call web jobs. Does anyone use web jobs, Azure web jobs? Okay, a few more people there. So, web jobs are kind of like schedule tasks, but on drugs. They're just amazing. They can do so much more. Again, they're really simple to set up and really simple to deploy. In Azure, if you're using Visual Studio, then you can plug into them and you can... Let me go in here. So, you can look at PDF generation. Web jobs, continuous. Okay. And so, you can go into them. You can view the status of them, so they have an inbuilt dashboard, so you can see if they're erroring. You can see the actual queues of data that they're processing. You can see if they're started or stopped. So, this is stopped because we haven't used it for a while. So, you can start it up, and then once it's started up, you can view the dashboard. Oh, apparently. A file name has not been provided. Yeah, that makes sense. Okay, so I won't show you that bit. But you can go in and you can view the dashboard, which is just really cool. You can replay events. So, if something failed and you didn't know why, you can replay it. You can attach your debugger to it. So, from Visual Studio, you can go in and you can say, attach debugger. You can add breakpoints in your code, and you can follow the whole web job through its execution, which is just amazing. So, if you've got problems, you can step through and you can see where it's going wrong. Essentially, they're just console apps, and you build it in Visual Studio. You've got the Azure SDK. Grab it into Visual Studio, build it, deploy it to your web apps, which is kind of what websites is now called. You can run them for free. On the free tier, you have limited amount of CPU per day or per, I think, per two hours. You can have like 10 minutes of CPU time or something. If you go to the paid version, then you can run them endlessly. You can scale them up, so you have massive, massive amounts of scale available. So, you can do parallel processing, multiple instances in parallel, so you can have huge, huge amounts of data going through those. Does anyone know? Troy Hunt, the Australian security expert guy. So, his have I been pwned, pawned thing website that he's got. So, he takes all these data breaches, puts them into his website, and you can check your email address to see whether your account's been compromised. He has like data dumps of 60 million records, that kind of thing. Puts them into his own message queue, sets up some web jobs, and he just scales them out. Adds multiple web jobs, scales them up, and just goes through this data for like the price of a cup of coffee, apparently. They're really, really powerful. Okay, so, finally we have reporting emails that we send out. So, every week we send emails to, you know, teachers who are interested in what their students are doing. And we use Azure to do this as well. So, we use the task scheduler, which essentially is a timer, and you can say when you want this thing to happen. You can use it to call an API. You can use it to do any kind of get request or post request. You can use it to drop messages into an Azure message queue. You can do quite a number of things with it. So, we set it up to run on a Sunday morning at about 3 a.m. And every Sunday morning, it drops a message into the message queue. And it's literally, you know, start reporting equals true. And that's it. So, it drops that in every Sunday. And then we have an Azure worker role, which is just a continuous, again, pretty much a console app just continuously waiting for something to happen. It checks the message queue every second. That's just a default. It could be, you know, every ten minutes or whatever, but every second. And then if it sees this message, it goes off and builds the list of reports to create. So, it goes to the database, see who wants reports, dumps it all into a temporary table, and then works through that. Each account that needs a report goes off, gets the data, puts it all together, puts it into a nice PDF, and then sends it through SendGrid, so the marketplace service that we use. And that whole process takes about one and a half hours, something like that. There's quite a lot of reports to send out. But the beauty of this is that it's on a completely separate instance to the API, to the website, because it's all kind of little bits of computing here and there in Azure. So, this whole bit doesn't impact the API and doesn't impact people playing the games. I mean, they're all coming from the same database, but it doesn't impact it in a way that you would if it was all running on one server. So, I did, like I said at the beginning, I did two things. I did two templates in Azure. In Umbraco, I actually created two pages, which I thought that was quite good. And I'm going to show you them. That's page number one. That's a login page. I do a bit of design in my spare time, as you can see. So, I think that looks pretty good. And then, where's my cursor? There it is. And then, page number two is there. Look at that. Thank you. It took me a while. I wasn't quite sure about the font size. But what this does, because we have to be quite specific about the order in which we publish stuff in Umbraco, because if the language item hasn't been published before the application, then it's going to fail. If a game doesn't have a skill type, which is a different content type, so if that skill type isn't in the database, then the game publish will fail. So, we have to order it in a way. So, this is basically a custom page. And you click publish. And it just publishes everything. And it takes about nine and a half hours. So, it's quite a long process. But it's 23,000 bits of data in there. So, it's going through quite a lot. But it does it in that kind of order that we need so that everything just has the right bits of property. So, the right bits of data to publish into our custom database. And to be honest, I've only used this a couple of times, which is a bit disappointing. Because I think it's probably my best piece of work so far. But it's just using Angular to go through this kind of list. And then just each one sending a publish request to an API. And then just bringing back the result. Moving on to the next one. So, that publish request is just a standard Umbraco publish request. Which then triggers our intercepted publish. And then off it goes. I'm not going to publish everything. Because we haven't got time. So, we needed to control the order of publish. We don't want to use the back office. Because then it times out. We don't want to know where you got to and that kind of thing. And we wanted the ability to point it at a certain environment. And publish the entire set of data in one click. Which just worked really well. And then, like I said, all we're doing is using a content service to do save and publish with status. Passing the content item that we want to publish. And then that's it. If it works, then we send a result and say brilliant. And then it moves on to the next one. If it doesn't, we send a result and say failed. So, finally, I just wanted to share a couple of pain points. Well, a few pain points, actually. Because, you know, it wasn't a perfect project. But we learned a lot through it. So, cause was one of the biggest pain points that we had. Because all of these games are calling our API through a browser. So, we had to put cause headers on pretty much everything. We had to put them on the API request themselves. We had to put them into the image processor. So, we're using image processor to resize images. We put a custom resizer in there. But we had to intercept in image processor. It adds headers to the image as it publishes it out. So, we had to put custom cause headers into the images within image processor. To get them to then be allowed within the games. We also had to use... Put cause headers in the blob storage. So, the Azure storage and the CDN storage. To allow these requests to happen. And that... There's a few lines of codes that you literally run once. And it will add those cause headers into your Azure storage. So, that wasn't too painful. But the whole thing, just for quite a while, just bugged us with lots of problems. We had to resize images to unknown dimensions. So, you're playing these games on a phone. You're playing them on a tablet, on a desktop. We don't have specific kind of... This is a 300 pixel wide image. This is a 200 pixel wide image. So, we had to add something custom into image processor. So, we could say we want a small or a medium or a large image. And that would just be a percentage. Rather than a specific pixel width. And so, we had to take James' image processor code. And write a custom resizer into that. Which was fun. And I sent that to James and said, look what I did. And he went, it's not quite right, is it? But it worked. And then I just had to move on. I didn't have time to go through it and rewrite it properly. As you do. Lots of effort behind the scenes to set up all those doc types. To set up all the magnificent switch statements. And do each doc type have its own code. And its own custom kind of intercept method. And its own custom data type in our database. So, each time you add a new doc type. Each time you add a property to a doc type that already exists. You have to follow it all the way through to the API. And put all those pieces of data in there. So, there's a lot of maintenance in that sense. But like I say, it makes the content creation part really simple and really easy. So, it's worth it. We couldn't get errors to display in Umbraco user interface. So, when you publish something, if our intercept publish failed. And we couldn't put it into our database for whatever reason. We couldn't bubble that error back up into the red box at the bottom. There's some problem happening. The only way we could do that was essentially turn custom errors off. And just throw a really nasty error back onto the page. There's no way to get your errors from that intercept code back into the database. And then just back into the angular to show it in the back office. As I've already said, we had to publish everything in a specific order. So, that was a little bit painful. But I wrote that page to deal with that. The file upload hack. It works, but it wasn't great. We had this problem. I don't know if anyone else had this problem. With Windows Server 2012. R2. Release 2. It kept restarting the Umbraco application. Because we were doing long running publishes on there. And there was some bug in Windows Server that would just restart when a process was going for quite a long time. And the Umbraco guys dealt with Microsoft to get a patch for that. But while that was happening, each time we released Umbraco to our Azure instance. We had to remote desktop on. Change the operating system. Back to 2012. It was plain. Then restart the whole thing. And then run it from there. So, it was a, you know. We don't have to do that now. Because that patch has been up there. But we had to do that manual work around every time we deployed. Which was a bit of a pain. And then, oh, you get two at once there. You get limited processing time on the web jobs free tier. So, that PDF thumbnail process. It would. To start off with. Like I said, you only had something like 20 minutes per three hours or something like that. I can't remember the exact details. But if you move to the paid version, it's unlimited. So, that was. It worked out fine. And in reality, we only need to switch that on every now and again. Because we only change the PDFs in batches. So, when we change a batch of PDFs. We start the service. It runs in the background. By the time it's done, we've used the free minutes. But that's fine. One final thing with web jobs. They don't take into account the app.config web transforms. So, you can add web transforms using something like slow cheater in Visual Studio. And that works fine for, say, a worker role. The Umbraco worker roles. And they will manage the web transforms when you publish them. But the web jobs completely ignore that. So, every time you deploy the web job. You have to go in manually. You know, change the connection strings and that kind of stuff. So, that's a bit of a pain. But again, we don't really touch that. It works and it runs. And we don't really touch that very often. So, we don't need to worry about that too much. But, yeah. Leaving on a down point there. Thank you. Do we have any questions? No? Good. Okay. Thanks a lot. Thank you. Thank you. Thank you. Thank you.