[tests] Stand up iOS arm runtime tests and enable tracing/eventpipe tests by mdh1418 · Pull Request #79169 · dotnet/runtime

mdh1418 · 2022-12-02T19:45:24Z

On iOS, runtime tests were previously only stood up on x64 (Simulator) in #43954.
This PR looks to extend the runtime testing coverage to iOS devices and arm simulators by doing the following:

Provide AOT'd assemblies to the AppleAppBuilder in the runtime build flow following the conventions of library tests (previously no assemblies were provided to the AppleAppBuilder)
Default RunAOTCompilation and MonoForceInterpreter properties in the runtime build flow for iOS
Enable diagnostics_tracing for iOS runtime tests
Reenable runtime tracing/eventpipe tests on iOS not involving subprocesses
Tweak ProcessInfo and ProcessInfo2 tests to pass on iOS
Skip IpcTraceTest.EnsureCleanEnvironment as ProcessManager.GetProcessInfos is not supported on iOS
Align AppleAppBuilder parameter order with the order of definition in the task

Fixes: #81233

ghost · 2022-12-02T19:45:31Z

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes #74891

On iOS, runtime tests were previously only stood up on x64 (Simulator) in #43954.
This PR looks to extend the runtime testing coverage to iOS devices and arm simulators by doing the following:

Provide AOT'd assemblies to the AppleAppBuilder in the runtime build flow following the conventions of library tests (previously no assemblies were provided to the AppleAppBuilder)
Default RunAOTCompilation and MonoForceInterpreter properties in the runtime build flow for iOS
Enable diagnostics_tracing for iOS runtime tests
Reenable runtime tracing/eventpipe tests on iOS not involving subprocesses
Tweak ProcessInfo and ProcessInfo2 tests to pass on iOS
Skip IpcTraceTest.EnsureCleanEnvironment as ProcessManager.GetProcessInfos is not supported on iOS
Align AppleAppBuilder parameter order with the order of definition in the task

Author:	mdh1418
Assignees:	-
Labels:	`area-Infrastructure-mono`, `runtime-mono`
Milestone:	-

naricc · 2022-12-16T21:21:15Z

I'm a little confused here; what happens if we are running iOSSimulator and both RunAOTCompilation and MonoForceInterpreter are true? Does this change how iOSSimulator tests are run?

For iOSlike devices and arm based simulators, AOT is needed for the tests to build. (At least that was the case for library tests, which motivated this PR #76551) When setting RunAOTCompilation and MonoForceInterpreter to true, what will happen is AOT will be ran, and if anything goes wrong, interp will be the fallback instead of JIT (which is important because JIT does not work on iOSlike platforms).
I believe the RunAOTCompilation is necessary here for arm architectures, but we can exclude x64 if thats preferred.

mdh1418 · 2022-12-16T23:30:33Z

/azp run runtime-ioslike,runtime-ioslikesimulator

azure-pipelines · 2022-12-16T23:30:44Z

Azure Pipelines successfully started running 2 pipeline(s).

mdh1418 · 2022-12-19T16:56:00Z

/azp run runtime-ioslike,runtime-ioslikesimulator

azure-pipelines · 2022-12-19T16:56:14Z

Azure Pipelines successfully started running 2 pipeline(s).

mdh1418 · 2022-12-19T18:05:31Z

/azp run runtime-ioslike,runtime-ioslikesimulator

azure-pipelines · 2022-12-19T18:05:44Z

Azure Pipelines successfully started running 2 pipeline(s).

naricc · 2022-12-19T22:00:41Z

/azp run runtime-ioslike,runtime-ioslikesimulator

azure-pipelines · 2022-12-19T22:02:16Z

Azure Pipelines successfully started running 2 pipeline(s).

naricc · 2022-12-20T15:45:28Z

/azp run runtime-ioslike

azure-pipelines · 2022-12-20T15:45:39Z

Azure Pipelines successfully started running 1 pipeline(s).

naricc · 2023-01-06T18:35:43Z

/azp run runtime-ioslike,runtime-ioslikesimulator

azure-pipelines · 2023-01-06T18:35:57Z

Azure Pipelines failed to run 2 pipeline(s).

kotlarmilos · 2023-04-13T09:49:59Z

/azp run runtime-ioslike,runtime-ioslikesimulator

azure-pipelines · 2023-04-13T09:50:23Z

Azure Pipelines successfully started running 2 pipeline(s).

kotlarmilos · 2023-04-13T12:53:15Z

The failure shouldn't be related. It has been tracked in dotnet/arcade#11683.

mdh1418

Just a bunch of nits and Jeremy's suggestion for the test grouping, but looks good overall to me!

Would approve, but the approve button is greyed out for me because I was the original PR author

steveisok

Looks good!

kotlarmilos · 2023-04-14T15:08:07Z

/azp run runtime-ioslike,runtime-ioslikesimulator

azure-pipelines · 2023-04-14T15:08:22Z

Azure Pipelines successfully started running 2 pipeline(s).

kotlarmilos · 2023-04-17T08:38:33Z

/azp run runtime-ioslikesimulator

azure-pipelines · 2023-04-17T08:38:44Z

Azure Pipelines successfully started running 1 pipeline(s).

mdh1418 · 2023-04-17T15:22:41Z

/azp run runtime-ioslikesimulator

azure-pipelines · 2023-04-17T15:22:56Z

Azure Pipelines successfully started running 1 pipeline(s).

kotlarmilos · 2023-04-18T03:14:12Z

/azp run runtime-ioslike,runtime-ioslikesimulator

azure-pipelines · 2023-04-18T03:14:27Z

Azure Pipelines successfully started running 2 pipeline(s).

kotlarmilos · 2023-04-18T14:06:12Z

Failures shouldn't be related. The TCP failure is known issue dotnet/arcade#11683.

kotlarmilos · 2023-04-18T17:46:03Z

The failures are known #84979 and #84995.

mdh1418 added area-Infrastructure-mono runtime-mono specific to the Mono runtime labels Dec 2, 2022

mdh1418 requested review from akoeplinger, fanyang-mono, lateralusX, naricc and steveisok December 2, 2022 19:45

mdh1418 requested a review from marek-safar as a code owner December 2, 2022 19:45

ghost assigned mdh1418 Dec 2, 2022

naricc reviewed Dec 16, 2022

View reviewed changes

This was referenced Dec 17, 2022

Tracking issue for CI build timeouts #76454

Closed

Precondition failure: File has not had execution verified #79439

Closed

[wasm] Library tests failing during linking for AOT - SIGKILL #79569

Closed

mdh1418 commented Dec 19, 2022

View reviewed changes

Comment thread eng/pipelines/extra-platforms/runtime-extra-platforms-ioslike.yml Outdated

build-analysis Bot mentioned this pull request Dec 20, 2022

GenerateHWIntrinsicTests_Arm/project.assets.json already exists #79843

Closed

build-analysis Bot mentioned this pull request Jan 6, 2023

TestResourceManagerIsSafeForConcurrentAccessAndEnumeration fails intermittently in CI #80277

Closed

naricc requested a review from vargaz as a code owner January 6, 2023 20:53

Resolve PR comments

cbccac5

mdh1418 commented Apr 13, 2023

View reviewed changes

fanyang-mono reviewed Apr 13, 2023

View reviewed changes

Comment thread eng/pipelines/extra-platforms/runtime-extra-platforms-ioslikesimulator.yml Outdated

Comment thread src/tests/Common/CLRTest.Execute.Bash.targets

Comment thread src/tests/Common/Coreclr.TestWrapper/MobileAppHandler.cs Outdated

steveisok approved these changes Apr 14, 2023

View reviewed changes

Resolve PR comments

3645fee

Test iossimulator_arm64 on rolling builds

f220b4e

build-analysis Bot mentioned this pull request Apr 17, 2023

[6.0] Android test failure: Failed to install mobile app #83422

Open

Revert test iossimulator_arm64 on rolling builds

ec4988b

steveisok approved these changes Apr 18, 2023

View reviewed changes

Comment thread eng/pipelines/extra-platforms/runtime-extra-platforms-ioslikesimulator.yml Outdated

Remove isRollingBuild condition

d6fd78a

kotlarmilos mentioned this pull request Apr 18, 2023

Could not load file or assembly 'Microsoft.CodeAnalysis.NetAnalyzers #84995

Closed

kotlarmilos merged commit 790b14b into dotnet:main Apr 18, 2023

kotlarmilos mentioned this pull request Apr 28, 2023

how can I run runtime tests with monovm on ios? #85160

Closed

ghost locked as resolved and limited conversation to collaborators May 18, 2023

mdh1418 deleted the ios_investigate_eventpipe_failures branch June 10, 2024 20:34

Conversation

mdh1418 commented Dec 2, 2022 • edited by naricc Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Dec 2, 2022

Uh oh!

naricc Dec 16, 2022

Choose a reason for hiding this comment

Uh oh!

mdh1418 Dec 19, 2022

Choose a reason for hiding this comment

Uh oh!

mdh1418 commented Dec 16, 2022

Uh oh!

azure-pipelines Bot commented Dec 16, 2022

Uh oh!

mdh1418 commented Dec 19, 2022

Uh oh!

azure-pipelines Bot commented Dec 19, 2022

Uh oh!

mdh1418 commented Dec 19, 2022

Uh oh!

azure-pipelines Bot commented Dec 19, 2022

Uh oh!

Uh oh!

naricc commented Dec 19, 2022

Uh oh!

azure-pipelines Bot commented Dec 19, 2022

Uh oh!

naricc commented Dec 20, 2022

Uh oh!

azure-pipelines Bot commented Dec 20, 2022

Uh oh!

naricc commented Jan 6, 2023

Uh oh!

azure-pipelines Bot commented Jan 6, 2023

Uh oh!

kotlarmilos commented Apr 13, 2023

Uh oh!

azure-pipelines Bot commented Apr 13, 2023

Uh oh!

kotlarmilos commented Apr 13, 2023

Uh oh!

mdh1418 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

steveisok left a comment

Choose a reason for hiding this comment

Uh oh!

kotlarmilos commented Apr 14, 2023

Uh oh!

azure-pipelines Bot commented Apr 14, 2023

Uh oh!

kotlarmilos commented Apr 17, 2023

Uh oh!

azure-pipelines Bot commented Apr 17, 2023

Uh oh!

mdh1418 commented Apr 17, 2023

Uh oh!

azure-pipelines Bot commented Apr 17, 2023

Uh oh!

kotlarmilos commented Apr 18, 2023

Uh oh!

azure-pipelines Bot commented Apr 18, 2023

Uh oh!

kotlarmilos commented Apr 18, 2023

mdh1418 commented Dec 2, 2022 •

edited by naricc

Loading

mdh1418 left a comment •

edited

Loading