chore(release): Update release testing section for new E2E testing script (facebook#3368)

kelset · web-flow · commit e4d09821bdfc · 2022-10-18T16:50:27.000+01:00
* remove E2E section until we'll re-add it post E2E setup

* remove mention of the E2E script

* rework the release testing page

* add mention of where to find the info to turn on new arch

* typo

* re-add section
diff --git a/website/contributing/how-to-contribute-code.md b/website/contributing/how-to-contribute-code.md
@@ -17,19 +17,19 @@ Please refer to the [Environment Setup](/docs/environment-setup) guide to setup
 
 After cloning React Native, open the directory and run `yarn` to install its dependencies.
 
-Then, you can run several commands:
+Now you are set up to run several commands:
 
 - `yarn start` starts the Metro packager server.
 - `yarn lint` checks the code style.
 - `yarn format` automatically formats your code.
-- `yarn test` runs the JavaScript test suite.
-- `yarn test --watch` runs an interactive JavaScript test watcher.
-- `yarn test <pattern>` runs JavaScript tests with matching filenames.
+- `yarn test` runs the Jest-based JavaScript test suite.
+  - `yarn test --watch` runs an interactive JavaScript test watcher.
+  - `yarn test <pattern>` runs JavaScript tests with matching filenames.
 - `yarn flow` runs the [Flow](https://flowtype.org/) typechecks.
-- `yarn flow-check-android` does a full Flow check over `*.android.js` files.
-- `yarn flow-check-ios` does a full Flow check over `*.ios.js` files.
+  - `yarn flow-check-android` does a full Flow check over `*.android.js` files.
+  - `yarn flow-check-ios` does a full Flow check over `*.ios.js` files.
+- `yarn test-typescript` runs the [TypeScript](https://www.typescriptlang.org/) typechecks.
 - `yarn test-ios` runs the iOS test suite (macOS required).
-- `node ./scripts/run-ci-e2e-tests.js --js --android --ios` runs the end-to-tend JavaScript, Android, and iOS tests.
 
 ## Testing your Changes
 
diff --git a/website/contributing/how-to-run-and-write-tests.md b/website/contributing/how-to-run-and-write-tests.md
@@ -48,6 +48,7 @@ Xcode also allows running individual tests through its Test Navigator. You can a
 If you are making modifications to `Podfile` configurations then there are Ruby tests that can verify these.
 
 To run the ruby tests:
+
 ```bash
 cd scripts
 sh run_ruby_tests.sh
@@ -79,16 +80,6 @@ Then, run the Android integration tests:
 ./scripts/run-android-local-integration-tests.sh
 ```
 
-### End-to-end Tests
-
-Finally, make sure end-to-end tests run successfully by executing the following script:
-
-```bash
-./scripts/test-manual-e2e.sh
-```
-
-This script will make you select to run the RNTester app (that lives within `packages/rn-tester`) or it will generate a fresh new project in `/tmp/RNTestProject`. Follow the step-by-step instructions from the script to test successfully if your changes.
-
 ## Writing Tests
 
 Whenever you are fixing a bug or adding new functionality to React Native, it is a good idea to add a test that covers it. Depending on the change you're making, there are different types of tests that may be appropriate.
diff --git a/website/contributing/release-testing.md b/website/contributing/release-testing.md
@@ -3,11 +3,20 @@ id: release-testing
 title: How to Test a Release
 ---
 
-## Test source in the release branch
-
 These steps needs to be followed by the release crew as part of the release process, to ensure that new versions published have a good level of stability.
 
-### Pre-requisites
+:::info
+
+An important element of testing source in the release branch is that this process needs to be preferably be done twice, on two separate machines - there are multiple scenarios in which something might fail on a machine but not on another. By running tests on two computers, we want to reach a step of double confirmation that everything works fine.
+:::
+
+## Pre-requisites
+
+:::note
+
+Currently, this flow can only be done on macOS machines.
+
+:::
 
 - Have a clone of `react-native` repo and be on the release branch (`0.XX-stable`).
 
@@ -19,14 +28,18 @@ These steps needs to be followed by the release crew as part of the release proc
   echo '{}' > .watchmanconfig
   ```
 
-- Have Android and iOS development environment set-up. Follow instructions for macOS/iOS and macOS/Android from the [Environment Setup](/docs/environment-setup) guide.
+- Have Android and iOS development environment set-up. Follow instructions for `React Native CLI quickstart` for macOS/iOS and macOS/Android from the [Environment Setup](/docs/environment-setup) guide.
 
   #### Additional steps for Android
 
   - Gradle should now install [the appropriate NDK](https://github.com/facebook/react-native/blob/main/template/android/build.gradle). Verify that you have in your path the `ANDROID_NDK` variable, pointing to it.
   - In case you are on macOS Catalina (or higher), you might also need to run `sudo xattr -r -d com.apple.quarantine /path/to/ndk` to avoid the e2e script to fail. (_That said, this should not happen anymore since from NDK 21 and higher the Android team started signing the NDK._)
 
-### Steps
+## Steps
+
+### Clean up the local state
+
+When testing locally, we want to ensure that we start from a clean slate to avoid caches polluting our testing.
 
 1. Delete `RNTester` and `RNTestProject` from your Android emulator and iOS simulator if leftover from previous test.
 2. Remove any temporary files from the `react-native` repo:
@@ -35,76 +48,73 @@ These steps needs to be followed by the release crew as part of the release proc
     git clean -fdx
    ```
 
+   For `main` branch, and versions of RN >=0.71, you can instead use `yarn test-e2e-local-clean`.
+
 3. Install dependencies:
 
    ```bash
     yarn install
-    pushd packages/rn-tester
-    pod install --repo-update
-    popd
    ```
 
-4. Use the `test-manual-e2e` script to test `RNTester` and the template app (`RNTestProject`):
+### Generating the projects
 
-   ```bash
-   # This will run you through the different variants in Test Dimensions table
-   ./scripts/test-manual-e2e.sh
-   ```
+The local testing for a release consist of running the [test project](https://github.com/facebook/react-native/tree/main/packages/rn-tester) of the react-native repository, `RNTester`, which contains an in-depth list of components implementations, and generating a fresh new project based on the local code, `RNTestProject`, that will simulate accurately how a `react-native init` project will behave.
 
-5. Turn on Hermes in the `RNTestProject` and ensures it builds successfully.
+To generate the the right project with the specific configuration desired, you can use the command
 
-   - Enable Hermes for Android:
+```bash
+yarn test-e2e-local [options]
+```
 
-     ```bash
-     # Update `/tmp/RNTestProject/android/app/build.gradle` to `enableHermes`
-     project.ext.react = [
-       enableHermes: true,  // clean and rebuild if changing
-     ]
+Followed by the wanted options:
 
-     # Clean and rebuild
-     /tmp/RNTestProject/android$ ./gradlew clean
-     /tmp/RNTestProject$ npx react-native run-android
-     ```
+```bash
+  --help          Show help                                            [boolean]
+  --version       Show version number                                  [boolean]
+  -t, --target      [choices: "RNTester", "RNTestProject"] [default: "RNTester"]
+  -p, --platform                    [choices: "iOS", "Android"] [default: "iOS"]
+  -h, --hermes                                         [boolean] [default: true]
+```
 
-   - Enable Hermes for iOS:
+#### Versions older than 71
 
-     ```bash
-     # Update `/tmp/RNTestProject/ios/Podfile` and then run `pod install`
-     use_react_native!(
-       :path => config[:reactNativePath],
-       # to enable hermes on iOS, change `false` to `true` and then install pods
-       :hermes_enabled => true
-     )
+You need to use the interactive script run you through the different variants in [Test Dimensions](#test-dimensions):
 
-     # Install pods and run
-     /tmp/RNTestProject/ios$ pod install
-     /tmp/RNTestProject$ npx react-native run-ios
-     ```
+```bash
+./scripts/test-manual-e2e.sh
+```
 
-### Test Dimensions
+This script will ask you to select which platform and which project you want to test, and then to execute a series of extra steps during the process. Bear in mind, when testing RNTester on Android, you need to start the Android emulator ahead of time or it will fail.
 
-Covered by running `test-manual-e2e.sh`, see [issue](https://github.com/facebook/react-native/issues/33015) about supporting those "manual" cases.
+## What to test?
 
-| Variant          | RNTester                 | Template App             |
-| ---------------- | ------------------------ | ------------------------ |
-| Android - JSC    | via `test-manual-e2e.sh` | via `test-manual-e2e.sh` |
-| Android - Hermes | via `test-manual-e2e.sh` | manual                   |
-| iOS - JSC        | via `test-manual-e2e.sh` | via `test-manual-e2e.sh` |
-| iOS - Hermes     | via `test-manual-e2e.sh` | manual                   |
+Aside from verifying that the building process is successful, once the app spawn by the script is up and running, we want to run a series of manual tests to ensure that some core functionalities work, like Fast Refresh and the Flipper debugger.
 
-**Note well:** Starting from RN 0.70, Hermes is turned on by default so for the template app JSC will need to be manually tested by switching off Hermes.
+In the `RNTester` you want to also play around with the app and try different components: some important onces are `Flatlist`, `Image` and the "New Architecture Component" (should be the very last one in the list).
 
-### What to test?
+### Test Dimensions
 
-Aside from verifying that the building process is successful, once the app spawn by `test-manual-e2e.sh` is up and running, we want to run a series of manual tests to ensure that some core functionalities work, like Fast Refresh and the Flipper debugger.
+To ensure that we cover the most use cases, we need to ensure we test all these different combination of configurations:
 
-In the `RNTester` you want to also play around with the app and try different components: some important onces are `Flatlist`, `Image` and the "New Architecture Component" (should be the very last one in the list).
+- RNTester + iOS + Hermes
+- RNTester + iOS + JSC
+- RNTester + Android + Hermes
+- RNTester + Android + JSC
+- RNTestProject + iOS + Hermes
+- RNTestProject + iOS + JSC
+- RNTestProject + Android + Hermes
+- RNTestProject + Android + JSC
+
+:::note
+
+Bear in mind that RNTester project is already onboarded in the new architecture. `RNTestProject` is not - new architecture mode needs to be [enabled](/docs/the-new-architecture/use-app-template#enable-the-new-architecture) and tested separately.
+:::
 
-An important element of testing source in the release branch is that this process needs to be preferably be done twice, on two separate machines - there are multiple scenarios in which something might fail on a machine but not on another. By running tests on two computers, we want to reach a step of "double confirmation" that everything works fine.
+## Testing pre-releases (RC) on production apps
 
-## Testing an RC on a project - checklist
+During the Release Candidate (RC) phase of a release cycle, we ask for the community to set as dependency in their apps the latest RC available and report in the related "Road to 0.XX" how it performs ([example](https://github.com/reactwg/react-native-releases/discussions/26)).
 
-If you are a [release tester](./release-roles-responsibilities#release-tester-responsibilities), the ask for you is to set as dependency in your app the latest RC available and report in the related "Road to 0.XX" how it went ([example](https://github.com/reactwg/react-native-releases/discussions/26)). To help provide the relevant information, we have prepared this template: you can copy/pasted it in a comment and fill it accordingly.
+To help provide the relevant information, we have prepared this template they can use as blueprint for what is important to test - they can copy/pasted it in a comment and fill it accordingly.
 
 ```markdown
 | Link to branch:              |                         |
