测试自动化概述
首先,问问自己是否真的需要使用浏览器。很有可能,在某个时候,如果您正在开发一个复杂的 Web 应用程序,您将需要打开一个浏览器并实际测试它。
然而,诸如 Selenium 测试之类的功能性最终用户测试运行成本很高。此外,它们通常需要大量基础设施才能有效地运行。一个好的规则是,始终问问自己,是否可以使用更轻量级的测试方法(如单元测试)或更低级别的方法来完成您要测试的内容。
一旦您确定自己正在进行 Web 浏览器测试,并且您的 Selenium 环境已准备好开始编写测试,您通常会执行三个步骤的某种组合
- 设置数据
- 执行一组离散的操作
- 评估结果
您会希望尽可能缩短这些步骤;大多数情况下,一两个操作就足够了。浏览器自动化有“不可靠”的声誉,但实际上,这是因为用户经常对它要求过多。在后面的章节中,我们将回到您可以用来缓解测试中明显的间歇性问题的技术,特别是在如何克服浏览器和 WebDriver 之间的竞争条件。
通过保持测试简短,并且仅在绝对没有其他选择时才使用 Web 浏览器,您可以进行许多测试,而不会有最小的故障。
Selenium 测试的一个显著优点是其固有的能力,可以从用户的角度测试应用程序的所有组件,从后端到前端。因此,换句话说,虽然功能测试的运行成本可能很高,但它们也会一次性涵盖大量业务关键部分。
测试要求
如前所述,Selenium 测试的运行成本可能很高。具体程度取决于您运行测试的浏览器,但从历史上看,浏览器的行为差异很大,以至于经常将针对多个浏览器进行交叉测试作为既定目标。
Selenium 允许您在多个操作系统上的多个浏览器上运行相同的指令,但是所有可能的浏览器、它们的不同版本以及它们运行的许多操作系统的枚举将很快成为一项不简单的任务。
让我们从一个例子开始
Larry 编写了一个网站,允许用户订购他们定制的独角兽。
一般工作流程(我们称之为“快乐路径”)如下
- 创建帐户
- 配置独角兽
- 将其添加到购物车
- 结帐并付款
- 提供关于独角兽的反馈
很可能会尝试编写一个宏大的 Selenium 脚本来执行所有这些操作 - 许多人都会尝试。 抵制诱惑! 这样做会导致测试 a) 耗时很长,b) 会受到一些关于页面渲染时间问题的常见问题的困扰,以及 c) 如果测试失败,它不会为您提供一种简洁、一目了然的方法来诊断出了什么问题。
测试此场景的首选策略是将其分解为一系列独立的、快速的测试,每个测试都有一个存在的“理由”。
让我们假设您想要测试第二步:配置您的独角兽。它将执行以下操作
- 创建帐户
- 配置一个独角兽
请注意,我们跳过了其余步骤,我们将在完成此步骤后在其他小的、独立的测试用例中测试工作流程的其余部分。
首先,您需要创建一个帐户。在这里您需要做出一些选择
- 您是否要使用现有帐户?
- 您是否要创建一个新帐户?
- 在开始配置之前,是否需要考虑此类用户的任何特殊属性?
无论您如何回答这个问题,解决方案都是将其作为测试的“设置数据”部分的一部分。如果 Larry 已经公开了一个 API,使您(或任何人)能够创建和更新用户帐户,请务必使用它来回答这个问题。如果可能,您希望仅在您拥有一个“手中”的用户之后才启动浏览器,您可以使用该用户的凭据登录。
如果每个工作流程的每个测试都以创建用户帐户开始,则每次测试的执行都会增加许多秒。调用 API 并与数据库对话是快速的“无头”操作,不需要打开浏览器、导航到正确的页面、单击并等待表单提交等昂贵的过程。
理想情况下,您可以使用一行代码处理此设置阶段,该代码将在启动任何浏览器之前执行
// Create a user who has read-only permissions--they can configure a unicorn,
// but they do not have payment information set up, nor do they have
// administrative privileges. At the time the user is created, its email
// address and password are randomly generated--you don't even need to
// know them.
User user = UserFactory.createCommonUser(); //This method is defined elsewhere.
// Log in as this user.
// Logging in on this site takes you to your personal "My Account" page, so the
// AccountPage object is returned by the loginAs method, allowing you to then
// perform actions from the AccountPage.
AccountPage accountPage = loginAs(user.getEmail(), user.getPassword());
# Create a user who has read-only permissions--they can configure a unicorn,
# but they do not have payment information set up, nor do they have
# administrative privileges. At the time the user is created, its email
# address and password are randomly generated--you don't even need to
# know them.
user = user_factory.create_common_user() #This method is defined elsewhere.
# Log in as this user.
# Logging in on this site takes you to your personal "My Account" page, so the
# AccountPage object is returned by the loginAs method, allowing you to then
# perform actions from the AccountPage.
account_page = login_as(user.get_email(), user.get_password())
// Create a user who has read-only permissions--they can configure a unicorn,
// but they do not have payment information set up, nor do they have
// administrative privileges. At the time the user is created, its email
// address and password are randomly generated--you don't even need to
// know them.
User user = UserFactory.CreateCommonUser(); //This method is defined elsewhere.
// Log in as this user.
// Logging in on this site takes you to your personal "My Account" page, so the
// AccountPage object is returned by the loginAs method, allowing you to then
// perform actions from the AccountPage.
AccountPage accountPage = LoginAs(user.Email, user.Password);
# Create a user who has read-only permissions--they can configure a unicorn,
# but they do not have payment information set up, nor do they have
# administrative privileges. At the time the user is created, its email
# address and password are randomly generated--you don't even need to
# know them.
user = UserFactory.create_common_user #This method is defined elsewhere.
# Log in as this user.
# Logging in on this site takes you to your personal "My Account" page, so the
# AccountPage object is returned by the loginAs method, allowing you to then
# perform actions from the AccountPage.
account_page = login_as(user.email, user.password)
// Create a user who has read-only permissions--they can configure a unicorn,
// but they do not have payment information set up, nor do they have
// administrative privileges. At the time the user is created, its email
// address and password are randomly generated--you don't even need to
// know them.
var user = userFactory.createCommonUser(); //This method is defined elsewhere.
// Log in as this user.
// Logging in on this site takes you to your personal "My Account" page, so the
// AccountPage object is returned by the loginAs method, allowing you to then
// perform actions from the AccountPage.
var accountPage = loginAs(user.email, user.password);
// Create a user who has read-only permissions--they can configure a unicorn,
// but they do not have payment information set up, nor do they have
// administrative privileges. At the time the user is created, its email
// address and password are randomly generated--you don't even need to
// know them.
val user = UserFactory.createCommonUser() //This method is defined elsewhere.
// Log in as this user.
// Logging in on this site takes you to your personal "My Account" page, so the
// AccountPage object is returned by the loginAs method, allowing you to then
// perform actions from the AccountPage.
val accountPage = loginAs(user.getEmail(), user.getPassword())
正如您可以想象的那样,UserFactory
可以扩展为提供诸如 createAdminUser()
和 createUserWithPayment()
之类的方法。关键是,这两行代码不会分散您对此测试最终目的的注意力:配置独角兽。
页面对象模型的复杂性将在后面的章节中讨论,但我们将在本文中介绍这个概念
您的测试应由在站点页面上下文中从用户的角度执行的操作组成。这些页面存储为对象,其中包含有关网页如何组成以及如何执行操作的特定信息,而测试人员不应过多关注这些信息。
您想要什么样的独角兽?您可能想要粉色的,但不一定。最近紫色非常受欢迎。她需要太阳镜吗?星星纹身?这些选择虽然困难,但却是您作为测试人员的主要关注点 - 您需要确保您的订单履行中心将正确的独角兽发送给正确的人,而这始于这些选择。
请注意,在该段落中,我们没有谈论按钮、字段、下拉列表、单选按钮或 Web 表单。您的测试也不应该谈论! 您希望像用户尝试解决他们的问题一样编写代码。这是执行此操作的一种方法(继续前面的示例)
// The Unicorn is a top-level Object--it has attributes, which are set here.
// This only stores the values; it does not fill out any web forms or interact
// with the browser in any way.
Unicorn sparkles = new Unicorn("Sparkles", UnicornColors.PURPLE, UnicornAccessories.SUNGLASSES, UnicornAdornments.STAR_TATTOOS);
// Since we are already "on" the account page, we have to use it to get to the
// actual place where you configure unicorns. Calling the "Add Unicorn" method
// takes us there.
AddUnicornPage addUnicornPage = accountPage.addUnicorn();
// Now that we're on the AddUnicornPage, we will pass the "sparkles" object to
// its createUnicorn() method. This method will take Sparkles' attributes,
// fill out the form, and click submit.
UnicornConfirmationPage unicornConfirmationPage = addUnicornPage.createUnicorn(sparkles);
# The Unicorn is a top-level Object--it has attributes, which are set here.
# This only stores the values; it does not fill out any web forms or interact
# with the browser in any way.
sparkles = Unicorn("Sparkles", UnicornColors.PURPLE, UnicornAccessories.SUNGLASSES, UnicornAdornments.STAR_TATTOOS)
# Since we're already "on" the account page, we have to use it to get to the
# actual place where you configure unicorns. Calling the "Add Unicorn" method
# takes us there.
add_unicorn_page = account_page.add_unicorn()
# Now that we're on the AddUnicornPage, we will pass the "sparkles" object to
# its createUnicorn() method. This method will take Sparkles' attributes,
# fill out the form, and click submit.
unicorn_confirmation_page = add_unicorn_page.create_unicorn(sparkles)
// The Unicorn is a top-level Object--it has attributes, which are set here.
// This only stores the values; it does not fill out any web forms or interact
// with the browser in any way.
Unicorn sparkles = new Unicorn("Sparkles", UnicornColors.Purple, UnicornAccessories.Sunglasses, UnicornAdornments.StarTattoos);
// Since we are already "on" the account page, we have to use it to get to the
// actual place where you configure unicorns. Calling the "Add Unicorn" method
// takes us there.
AddUnicornPage addUnicornPage = accountPage.AddUnicorn();
// Now that we're on the AddUnicornPage, we will pass the "sparkles" object to
// its createUnicorn() method. This method will take Sparkles' attributes,
// fill out the form, and click submit.
UnicornConfirmationPage unicornConfirmationPage = addUnicornPage.CreateUnicorn(sparkles);
# The Unicorn is a top-level Object--it has attributes, which are set here.
# This only stores the values; it does not fill out any web forms or interact
# with the browser in any way.
sparkles = Unicorn.new('Sparkles', UnicornColors.PURPLE, UnicornAccessories.SUNGLASSES, UnicornAdornments.STAR_TATTOOS)
# Since we're already "on" the account page, we have to use it to get to the
# actual place where you configure unicorns. Calling the "Add Unicorn" method
# takes us there.
add_unicorn_page = account_page.add_unicorn
# Now that we're on the AddUnicornPage, we will pass the "sparkles" object to
# its createUnicorn() method. This method will take Sparkles' attributes,
# fill out the form, and click submit.
unicorn_confirmation_page = add_unicorn_page.create_unicorn(sparkles)
// The Unicorn is a top-level Object--it has attributes, which are set here.
// This only stores the values; it does not fill out any web forms or interact
// with the browser in any way.
var sparkles = new Unicorn("Sparkles", UnicornColors.PURPLE, UnicornAccessories.SUNGLASSES, UnicornAdornments.STAR_TATTOOS);
// Since we are already "on" the account page, we have to use it to get to the
// actual place where you configure unicorns. Calling the "Add Unicorn" method
// takes us there.
var addUnicornPage = accountPage.addUnicorn();
// Now that we're on the AddUnicornPage, we will pass the "sparkles" object to
// its createUnicorn() method. This method will take Sparkles' attributes,
// fill out the form, and click submit.
var unicornConfirmationPage = addUnicornPage.createUnicorn(sparkles);
// The Unicorn is a top-level Object--it has attributes, which are set here.
// This only stores the values; it does not fill out any web forms or interact
// with the browser in any way.
val sparkles = Unicorn("Sparkles", UnicornColors.PURPLE, UnicornAccessories.SUNGLASSES, UnicornAdornments.STAR_TATTOOS)
// Since we are already "on" the account page, we have to use it to get to the
// actual place where you configure unicorns. Calling the "Add Unicorn" method
// takes us there.
val addUnicornPage = accountPage.addUnicorn()
// Now that we're on the AddUnicornPage, we will pass the "sparkles" object to
// its createUnicorn() method. This method will take Sparkles' attributes,
// fill out the form, and click submit.
unicornConfirmationPage = addUnicornPage.createUnicorn(sparkles)
现在您已经配置了独角兽,您需要继续执行第 3 步:确保它实际有效。
// The exists() method from UnicornConfirmationPage will take the Sparkles
// object--a specification of the attributes you want to see, and compare
// them with the fields on the page.
Assert.assertTrue("Sparkles should have been created, with all attributes intact", unicornConfirmationPage.exists(sparkles));
# The exists() method from UnicornConfirmationPage will take the Sparkles
# object--a specification of the attributes you want to see, and compare
# them with the fields on the page.
assert unicorn_confirmation_page.exists(sparkles), "Sparkles should have been created, with all attributes intact"
// The exists() method from UnicornConfirmationPage will take the Sparkles
// object--a specification of the attributes you want to see, and compare
// them with the fields on the page.
Assert.True(unicornConfirmationPage.Exists(sparkles), "Sparkles should have been created, with all attributes intact");
# The exists() method from UnicornConfirmationPage will take the Sparkles
# object--a specification of the attributes you want to see, and compare
# them with the fields on the page.
expect(unicorn_confirmation_page.exists?(sparkles)).to be, 'Sparkles should have been created, with all attributes intact'
// The exists() method from UnicornConfirmationPage will take the Sparkles
// object--a specification of the attributes you want to see, and compare
// them with the fields on the page.
assert(unicornConfirmationPage.exists(sparkles), "Sparkles should have been created, with all attributes intact");
// The exists() method from UnicornConfirmationPage will take the Sparkles
// object--a specification of the attributes you want to see, and compare
// them with the fields on the page.
assertTrue("Sparkles should have been created, with all attributes intact", unicornConfirmationPage.exists(sparkles))
请注意,测试人员仍然没有做任何事情,只是在此代码中谈论独角兽 - 没有按钮、没有定位器、没有浏览器控件。这种建模应用程序的方法使您可以保持这些测试级别命令不变,即使 Larry 下周决定他不再喜欢 Ruby-on-Rails 并决定使用最新的 Haskell 绑定和 Fortran 前端重新实现整个站点。
您的页面对象需要进行一些小的维护,以符合网站的重新设计,但这些测试将保持不变。 考虑到这个基本设计,您需要尽可能地以最少的浏览器操作步骤来完成您的工作流程。您的下一个工作流程将涉及将独角兽添加到购物车。您可能需要对这个测试进行多次迭代,以确保购物车正确地保持其状态:在您开始之前,购物车中是否有多于一只独角兽?购物车可以容纳多少只?如果您创建多个具有相同名称和/或功能的独角兽,它会崩溃吗?它只会保留现有的独角兽还是会添加另一个?
每次您完成工作流程时,您都希望尽量避免创建账户、以用户身份登录以及配置独角兽。理想情况下,您应该能够通过 API 或数据库创建一个账户并预先配置一个独角兽。然后,您只需以用户身份登录,找到 Sparkles,然后将她添加到购物车即可。
要自动化还是不自动化?
自动化总是好的吗?应该在什么时候决定自动化测试用例?
自动化测试用例并不总是好的。 有时,手动测试可能更合适。例如,如果应用程序的用户界面在不久的将来会发生很大变化,那么任何自动化都可能需要重写。此外,有时根本没有足够的时间来构建测试自动化。在短期内,手动测试可能更有效。如果应用程序的截止日期非常紧迫,目前没有可用的测试自动化,并且必须在该时间范围内完成测试,那么手动测试是最佳解决方案。