Different report final result when running on AS vs IntelliJ

[cwchan]

AppiumStudio version: 11.3.82
IntelliJ IDEA 2017.3 (Community Edition)
Build #IC-173.3727.127, built on November 27, 2017
JRE: 1.8.0_152-release-1024-b6 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Mac OS X 10.13.2

PROBLEM:
I recorded a 12 steps test case using AS. When I playback the steps using AS, the final result is a pass according to the report displayed at the end of execution.

Now, I copied Java code from AS to IntelliJ IDEA, ran it, and manually open the result HTML file. The final result, however, was a fail.

The failed step was “Wait for Element Presence”, or in Java code:
new WebDriverWait(driver, 10).until(ExpectedConditions.presenceOfElementLocated(By.xpath("//*[@text=‘Add Product’]")));

Passing on AS:

Failing on IntelliJ external IDE:
(To be attached as new user can only attach one picture in a post)

It appears that on external IDE, the driver tried to “Find Element” every 3 seconds. The first two attempts were unsuccessful and thus returned fail. This behavior, however, did not happen when running using AS.

Can someone please help me on this as it is kind of urgent. Thank you very much in advance.

[cwchan]

This is failed result from IntelliJ external IDE:

[cwchan]

This is the Java code generated by AS. The line in question is the last line in testUntitled() (e.g. new WebDriverWait(driver, 10)…):

//package <set your test package>;
import io.appium.java_client.remote.AndroidMobileCapabilityType;
import io.appium.java_client.android.AndroidDriver;
import io.appium.java_client.android.AndroidElement;
import io.appium.java_client.remote.MobileCapabilityType;
import io.appium.java_client.android.AndroidKeyCode;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.openqa.selenium.ScreenOrientation;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.By;
import org.junit.*;
import java.net.URL;
import java.net.MalformedURLException;

public class Untitled {
 private String reportDirectory = "reports";
 private String reportFormat = "xml";
 private String testName = "Untitled";
 protected AndroidDriver<AndroidElement> driver = null;

 DesiredCapabilities dc = new DesiredCapabilities();
 
 @Before
 public void setUp() throws MalformedURLException {
  dc.setCapability("reportDirectory", reportDirectory);
  dc.setCapability("reportFormat", reportFormat);
  dc.setCapability("testName", testName);
  dc.setCapability(MobileCapabilityType.UDID, "7f8b54d3");
  driver = new AndroidDriver<AndroidElement>(new URL("http://localhost:4723/wd/hub"), dc);
 }
 
 @Test
 public void testUntitled() {
  driver.findElement(By.xpath("//*[@contentDescription='Add Product']")).click();
  driver.findElement(By.xpath("(//*[@class='android.widget.LinearLayout' and ./parent::*[@class='android.widget.ScrollView']]/*[@class='android.widget.EditText'])[1]")).sendKeys("C0012");
  driver.findElement(By.xpath("(//*[@class='android.widget.LinearLayout' and ./parent::*[@class='android.widget.ScrollView']]/*[@class='android.widget.EditText'])[2]")).sendKeys("C0012");
  driver.findElement(By.xpath("//*[@text='Add/Update Image']")).click();
  try{Thread.sleep(5000);} catch(Exception ignore){}
  driver.pressKeyCode(AndroidKeyCode.BACK);
  driver.findElement(By.xpath("//*[@text='Photos']")).click();
  driver.findElement(By.xpath("//*[@id='dir_thumbnail' and ./following-sibling::*[@height>0 and ./*[@text='Pictures']]]")).click();
  driver.findElement(By.xpath("(//*[@id='media_grid']/*/*[@id='medium_thumbnail'])[1]")).click();
  driver.findElement(By.xpath("//*[@text='OK']")).click();
  new WebDriverWait(driver, 10).until(ExpectedConditions.presenceOfElementLocated(By.xpath("//*[@text='Add Product']")));
 }
 
 @After
 public void tearDown() {
     driver.quit();
 }
}

[Tom]

Hi

Are you instrumenting your application from Appium Studio?

Can you add a screenshot of the object spy when the script works from Appium Studio - please add the filter of your Xpath expression - I’d like to see the properties of the element

Thanks

Tom

[cwchan]

Dear Tom,

Thank you very much for the reply. I did not instrument my application and just record whatever on the screen.

Below please find the screenshot of the element in question. Basically, the app is uploading an image from local device to the app, we are waiting “Add Product” action bar title to appear.

Anyway, we discovered something…

It appeared that if we changed Implicit Wait time, then we will not get the failed step in external IDE. For instance:

    @Before
    public void setUp() throws MalformedURLException {
        dc.setCapability("reportDirectory", reportDirectory);
        dc.setCapability("reportFormat", reportFormat);
        dc.setCapability("testName", testName);
        dc.setCapability(MobileCapabilityType.UDID, "7f8b54d3");
        driver = new AndroidDriver<AndroidElement>(new URL("http://localhost:4723/wd/hub"), dc);
        driver.manage().timeouts().implicitlyWait(2, TimeUnit.MINUTES);
    }

We changed Implicit Wait time to 2 minutes and the final result was a PASS in IDE. There was no failed step.

This makes us wonder whether or not default Implicit Wait is 0 (meaning to wait until other timeout occurs) or non-zero?

Thanks.

[Tom]

Hi

Thanks for your updates

The default implicit wait time is in-fact zero

new WebDriverWait(driver, 10).until(ExpectedConditions.presenceOfElementLocated(By.xpath("//*[@text='Add Product']")));

This command tells Appium Studio to wait for your element for a maximum duration of 10 seconds

We can see in the report - that it tried to find the element 3 times, and finally succeded

The report is marked as failed because of the two tryouts, but the command finally passes

You can ignore these failed steps by using the executeScript command showReport

e.g

driver.executeScript("client:client.showReport(false)");
new WebDriverWait(driver, 10).until(ExpectedConditions.presenceOfElementLocated(By.xpath("//*[@text='Add Product']")));
driver.executeScript("client:client.showReport(true)");
driver.findElement(By.xpath("//*[@text='Add Product']")).click();

[cwchan]

Dear Tom,

Thank you for the solution and sorry for the late reply. I tried adding in the ExecuteScript command but received the following error:

System.InvalidOperationException : javascript error (javascript error. TypeError: client.showReport is not a function in <eval> at line number 1) (UnexpectedJavaScriptError)

Currently I am using Visual Studio Community + C#.

[Tom]

can you please share with me the line you are trying to execute?

[cwchan]

Hi Tom,

Sure. One of my automation tests is basically to delete all items in ListView. Each item has an “X” image button on upper right to allow user to tap on it to delete the respective item. Since the number of items is dynamic, I basically tell AS to perform item deletion until it can’t find any “X” image button (meaning all items are deleted from shopping cart).

        driver.ExecuteScript("client:client.showReport(false)");
        while(allDeleted == false){
            if (driver.FindElement(By.XPath("xpath=//*[@class='android.widget.ImageButton']")).Displayed)
            {
                driver.ExecuteScript("client:client.showReport(true)");
                driver.FindElement(By.XPath("xpath=//*[@class='android.widget.ImageButton']")).Click();
                new WebDriverWait(driver, TimeSpan.FromSeconds(10)).Until(ExpectedConditions.ElementExists(By.XPath("//*[@text='Delete?']")));
                driver.FindElement(By.XPath("xpath=//*[@text='Yes']")).Click();
                Thread.Sleep(4000);
                deleteCount = deleteCount + 1;
            }else{
                allDeleted = true;
            }   
        }

The third line (if (driver.FindElement(By.XPath("xpath=//*…) will attempt to find “X” image button and will return a failed step if the button cannot be found. Since the line is just a condition checking, my intention is to exclude it from reporting by using:

driver.executeScript("client:client.showReport(false)")

as suggest by you. As mentioned, my Visual Studio returns the following error on the above line:

System.InvalidOperationException : javascript error (javascript error. TypeError: client.showReport is not a function in <eval> at line number 1) (UnexpectedJavaScriptError)

Thanks.

[pmaristani]

Hi,

got the solution?

[Tom]

Hi

sorry for the delayed response

I’ve opened an issue on your request, seems like we had an issue with the showReport functionality, for the time being - i could offer a different solution

driver.executeScript('client:client.setShowPassImageInReport(false)");

this will option will cancel gathering images for steps that were passed, and only attach images for failed steps