Tests execution using Azure Functions

Spread the love

Idea

Let’s imagine a situation that you have to run a bunch of tests before pushing your hot-fix to production. CI machine is currently busy and you can’t wait. Azure Functions come with help!

Retrieving types

Firstly, we need a function that retrieves test type names from a dll. The function is invoked by HttpTrigger and adds all retrieved types to an output queue.

In line 22 you can see that fileName is passed as a part of a header – you can of course use a body. Then we need an identifier that allows us recognize to which .dll file the executed test belongs to (let’s imagine that we execute 3 similar dlls one by one). Let’s focus on line 26 a little bit longer. It turned out that test dll file must be located in the same place as an execution environment. The reason behind that was the method File.Exists() that is invoked inside xunit.runner (used for execution). I decided to upload test dll to a storage where AzureFunctions app is located. I didn’t find better solution to get a ‘true’ value from that method (if you know different solution please share it). So behind that ‘testDllDirectory’ is ‘D:\home\site\projectdlls’ by me -> ‘projectdlls’ is a folder in my AzureFuntions’ storage, where a compiled solution is stored. Next you can see that all my test types are classes and contain phrase ‘Test’ in their names. In line 34 retrieved types are passed into a queue.

Tests execution

Ok, our test types are currently in queue and we need a next function that executes tests. The following function takes information about tests from the queue, executes them and puts results to a result table.

Line 17 specifies a handler for resolving an assembly. Without it the xunit runner throws an exception. It happens during assembly loading (if you need more information go to this and this). One more thing an assembly resolve handler needs, is a location with utilized dlls. ‘DLL_Location’ refers to ‘D:\home\site\wwwroot\bin\’. There is an initialization of ManualResetEvent that indicates when execution is completed. Then a runner for selected test dll is created. After that handlers are assigned for different test results and the runner is started. Of course it is not a best idea to e.g. end function after error without logging some details but it is only proof of concept and everyone can add a better strategy :). Lines 30-33 contain a workaround – it turned out that despite the fact OnExecutionComplete is invoked the runner may not be disposable.

Implementation and configuration

Full implementation is available here. Moreover you have to remember to setup 2 variables in applications settings and connection to storage where a queue and a result table will be located.

Automation using LogicApps

I didn’t forget about an automation. Let’s create a Logic App.
First of all you have to create a connection to a storage where the queue and the result table will be stored. Then add a condition matching the test names and their file extension. ‘If false’ is left empty because another files can be ignored. When conditions are met the RetrieveTypes function is invoked. File name is passed into headers along with a secret key for our function (you can find it in Function app settings).

The flow is setup so now all you need to do is upload compiled solution (e.g. by Microsoft Azure Storage Explorer).

Summary

You have to remember about limitations. When a consumption plan is used then the function execution time is limited to 5 minutes. It means that one test class can’t take longer than 5 minutes to execute. Of course we can deal with it by splitting it into 2 or more classes.

The best thing is that omitting time for file recognition, etc., ALL TESTS WILL BE EXECUTED IN MAXIMAL 5 MINUTES!

2 thoughts on “Tests execution using Azure Functions

  1. Looks like very interesting use case for Azure Functions, haven’t seen anything similar before.

    The only confusion I have is location of tested DLLs. I would assume, test runner is always provisioned and just waiting for “test requests”, however it depends on test DLLs uploaded locally. As a result, you have to provision Function every time you build DLLs…
    – Maybe you could store DLLs in Blob Storage, since you have Storage Account already
    – I would also trigger test function not via HttpRequest but Blob trigger, every time new DLL is uploaded.

    1. When you have one test dll it seems to be that your idea with ‘always provisioned test runner’ will works. On the other hand, what’s happen when you have a few? It is good point to rethink during own implementation.
      -I’m not sure if it is possible to provide ‘normal’ path (e.g. ‘D:\home\site\wwwroot\bin\temp.test.dll’) to file instead of Uri(e.g. ‘https://storagesample.blob.core.windows.net/mycontainer/temp.test.dll’) in Blob Storage. I mentioned above that I tried to use the URI way but I always got an exception from XUnit.AssemblyRunner. It was caused by inner usage of ‘File.Exists’ that always returns false for Uri path.

Leave a Reply

Your email address will not be published. Required fields are marked *