Description Kristof Szabados 2020-11-26 08:41:53 EST

Sometimes tests running in CI systems, do find problems ... that is why we have them, sometimes they fail because of circumstantial reasons unrealated to their purpose.
However after the failure, when investigating the issue ... it does matter how much information is available to the user/developer/person doing the debugging work.

Titan is in a unique position, where it is possible to maximize the information available automatically for the user.
What could be possible:
1)
Using XCoverage to drive the CI processes, when it notices a test failing ... it could re-execute the same test 2-3 more times.

This way next morning, when checking the log, it would be immediately known, if the test was failing consistently for the same reason (and so is most probably reproducible) or inconsistently ( most probably related to some environmental factor)

2)
Having control of the configuration file, it would be possible to run the test once with all logging options enabled ... to have more information in the logs.

3)
Using the version handling system it might be possible to identify which exact commit caused the test to fail.
And as Titan already supports an automatic refactoring feature, which is able to insert extra log statements into the code (logging all visible variables)...
It would be possible to create a version of the code, where before each changed line, the values of all visible variables would be logged ... do a compilation and execution.

---

This would also mean, that by default the tests could run with few log statements in the code, reducing logging settings in the configuration file ... optimized for runtime performance.
And the CI system could automatically enhance all of these settings to extract as much information as possible about the issue ... reducing the time needed to debug it.