Let’s imagine a situation that you have to run a bunch of tests before pushing your hot-fix to production. CI machine is currently busy and you can’t wait. Azure Functions come with help!
Firstly, we need a function that retrieves test type names from a dll. The function is invoked by HttpTrigger and adds all retrieved types to an output queue.
In line 22 you can see that fileName is passed as a part of a header – you can of course use a body. Then we need an identifier that allows us recognize to which .dll file the executed test belongs to (let’s imagine that we execute 3 similar dlls one by one). Let’s focus on line 26 a little bit longer. It turned out that test dll file must be located in the same place as an execution environment. The reason behind that was the method File.Exists() that is invoked inside xunit.runner (used for execution). I decided to upload test dll to a storage where AzureFunctions app is located. I didn’t find better solution to get a ‘true’ value from that method (if you know different solution please share it). So behind that ‘testDllDirectory’ is ‘D:\home\site\projectdlls’ by me -> ‘projectdlls’ is a folder in my AzureFuntions’ storage, where a compiled solution is stored. Next you can see that all my test types are classes and contain phrase ‘Test’ in their names. In line 34 retrieved types are passed into a queue.
Ok, our test types are currently in queue and we need a next function that executes tests. The following function takes information about tests from the queue, executes them and puts results to a result table.
Line 17 specifies a handler for resolving an assembly. Without it the xunit runner throws an exception. It happens during assembly loading (if you need more information go to this and this). One more thing an assembly resolve handler needs, is a location with utilized dlls. ‘DLL_Location’ refers to ‘D:\home\site\wwwroot\bin\’. There is an initialization of ManualResetEvent that indicates when execution is completed. Then a runner for selected test dll is created. After that handlers are assigned for different test results and the runner is started. Of course it is not a best idea to e.g. end function after error without logging some details but it is only proof of concept and everyone can add a better strategy :). Lines 30-33 contain a workaround – it turned out that despite the fact OnExecutionComplete is invoked the runner may not be disposable.
Implementation and configuration
Full implementation is available here. Moreover you have to remember to setup 2 variables in applications settings and connection to storage where a queue and a result table will be located.
Automation using LogicApps
I didn’t forget about an automation. Let’s create a Logic App.
First of all you have to create a connection to a storage where the queue and the result table will be stored. Then add a condition matching the test names and their file extension. ‘If false’ is left empty because another files can be ignored. When conditions are met the RetrieveTypes function is invoked. File name is passed into headers along with a secret key for our function (you can find it in Function app settings).
The flow is setup so now all you need to do is upload compiled solution (e.g. by Microsoft Azure Storage Explorer).
You have to remember about limitations. When a consumption plan is used then the function execution time is limited to 5 minutes. It means that one test class can’t take longer than 5 minutes to execute. Of course we can deal with it by splitting it into 2 or more classes.
The best thing is that omitting time for file recognition, etc., ALL TESTS WILL BE EXECUTED IN MAXIMAL 5 MINUTES!