How can I get more granular Testresults in AWS-Device Farm? [Appium node.js]

Question

I run Appium node.js tests on AWS Device Farm. I would like to get granular Test results shown in Device Farm, but I always get one "Tests Suite" result which inlcudes all tests. So if one small test failes the whole Test Suite fails. I read in the Device Farm Docs that in a Standard Environment more granular results will be displayed, but I am not sure how to switch or use standard environment. I asume it has something to do with the YAML File as the possibility to select between standard or custom environment is not longer given on the Device Farm UI.

This is my current YAML File:

version: 0.1

# Phases are collection of commands that get executed on Device Farm.
phases:
  # The install phase includes commands that install dependencies that your tests use.
  # Default dependencies for testing frameworks supported on Device Farm are already installed.
  install:
    commands:
      # By default, Appium server version used is 1.7.2.
      # You can switch to an alternate supported version from 1.6.5, 1.7.1, 1.7.2, 1.8.0 , 1.8.1, 1.9.1 by using a command like "avm 1.7.1"
      # OR
      # To install a newer version of Appium use the following commands:
      - export APPIUM_VERSION=1.9.1
      - avm $APPIUM_VERSION
      - ln -s /usr/local/avm/versions/$APPIUM_VERSION/node_modules/.bin/appium  /usr/local/avm/versions/$APPIUM_VERSION/node_modules/appium/bin/appium.js

      # By default the node version installed is 11.4.0
      # you can switch to an alternate node version using below command.
      # - nvm install 10.13.0

      # Unpackage and install the node modules that you uploaded in the test phase.
      - echo "Navigate to test package directory"
      - cd $DEVICEFARM_TEST_PACKAGE_PATH
      - npm install *.tgz

  # The pre-test phase includes commands that setup your test environment.
  pre_test:
    commands:
      # We recommend starting appium server process in the background using the command below.
      # Appium server log will go to $DEVICEFARM_LOG_DIR directory.
      # The environment variables below will be auto-populated during run time.
      - echo "Start appium server"
      - >-
        appium --log-timestamp --device-name $DEVICEFARM_DEVICE_NAME
        --platform-name $DEVICEFARM_DEVICE_PLATFORM_NAME --app $DEVICEFARM_APP_PATH
        --automation-name UiAutomator2 --udid $DEVICEFARM_DEVICE_UDID
        --chromedriver-executable $DEVICEFARM_CHROMEDRIVER_EXECUTABLE  >> $DEVICEFARM_LOG_DIR/appiumlog.txt 2>&1 &

      - >-
        start_appium_timeout=0;
        while [ true ];
        do
            if [ $start_appium_timeout -gt 60 ];
            then
                echo "appium server never started in 60 seconds. Exiting";
                exit 1;
            fi;
            grep -i "Appium REST http interface listener started on 0.0.0.0:4723" $DEVICEFARM_LOG_DIR/appiumlog.txt >> /dev/null 2>&1;
            if [ $? -eq 0 ];
            then
                echo "Appium REST http interface listener started on 0.0.0.0:4723";
                break;
            else
                echo "Waiting for appium server to start. Sleeping for 1 second";
                sleep 1;
                start_appium_timeout=$((start_appium_timeout+1));
            fi;
        done; 

  # The test phase includes commands that run your test suite execution.
  test:
    commands:
      # Go into the root folder containing your source code and node_modules
      - echo "Navigate to test source code"
      # Change the directory to node_modules folder as it has your test code and the dependency node modules.
      - cd $DEVICEFARM_TEST_PACKAGE_PATH/node_modules/*

      - echo "Start Appium Node test"
      # Enter the command below to start the tests . The comamnd should be similar to what you use to run the tests locally.
      # For e.g. assuming you run your tests locally using command "node YOUR_TEST_FILENAME.js.", enter the same command below:
      - npm run test:android

  # The post test phase includes are commands that are run after your tests are executed.
  post_test:
    commands:

# The artifacts phase lets you specify the location where your tests logs, device logs will be stored.
# And also let you specify the location of your test logs and artifacts which you want to be collected by Device Farm.
# These logs and artifacts will be available through ListArtifacts API in Device Farm.
artifacts:
  # By default, Device Farm will collect your artifacts from following directories
  - $DEVICEFARM_LOG_DIR``` 

Answer 1

AWS Device Farm's standard mode is independent of YAML file. It's a setting that is configured in the "Configure" step when you schedule a run through the console or via CLI through the ScheduleRun API. Currently, AWS Device Farm does not support Appium Node in standard mode, which means that the granular reporting you are seeking is not available.

If you have further questions, you can head over to the AWS Device Farm Forums for additional assistance from their engineering team.

Andy