An XCResult is a bundle of test results created by Xcode. It contains detailed debugging and screenshots. As with all “bundles” on macOS, it’s actually a folder. As such, standard means of storing CI results do not apply. To add to the difficulty of figuring out how to store this on a particular CI, Apple changes the format of this output every couple of years; there may have been a lot of chatter on the subject a few years ago, but it’s worthless now. I was unable to find any help and instead had to cobble together a solution from a couple different places and my head. The solution is no more than a few lines of code, but it took a few days to arrive at. This is the Rubik’s cube of my job that I love.
- run:
name: Zip xcresult bundle
working_directory: build/Logs/Test/
command: zip -r testoutput.zip ./Run*/
when: always
- store_artifacts:
name: Store xcresult zip
path: build/Logs/Test/testoutput.zip
destination: xcresult.zip
The difficulties were the following:
- I didn’t know much of anything about storing artifacts or test results (there is a
store_test_results
command, but that doesn’t apply to XCResults). - Figuring out where the xcresult bundle lives; we don’t have much visibility into the CI file system so this was a lot of spraying, praying, and log parsing.
- The bundle directory is dynamic (includes a timestamp) so a wildcard would have to be used to match. Unfortunately, Circle’s
working_directory
parameter doesn’t support wildcards; I’d have to include it in the actual zip command. - CircleCI defaults
run
steps to only trigger if all previous steps are successful (contrast that to the commands that store things, which always run). If a test case failed, the zip file wouldn’t be created; the step must explicitly be set to always run.
I now get a zip file containing the bundle, which macOS is smart enough to automatically extract. A double-click later and I’m in! Note: This result was a smoke test; we’ve got a whole bunch more than 12 tests ?
Leave a Reply