How to run Magium tests inside of a Docker container

There is a little project I am working on that involves Magium, Docker, and awesome sauce. The sauce is still sitting in an underground cave, covered in salt, but there are parts of it that could have some eyes on it. The first part is that I now have a Docker container available for running Magium tests under. It uses the selenium/standalone-* Docker containers but I intend to have support for other containers shortly. The containers have PHP, Selenium Server, and the appropriate driver installed and ready to go. It uses Xvfb to run the browser.

There are three steps to run it. (If you have some trouble, feel free to open an issue on the Magium Github issue tracker)

First, place a new Dockerfile at the base of your Magium tests. It should look like this:

FROM magium/clairvoyant-chrome-php-7.0

COPY . /magium/

Composer is installed so if you need to do this you may need to have your Dockerfile look like this:

FROM magium/clairvoyant-chrome-php-7.0

COPY . /magium/
RUN composer update -d /magium

Then you need to build your Docker image

docker build -t my-tests .

Then run it

 docker run -ti -e "MAGIUM_EXEC=/magium/vendor/bin/phpunit /magium/tests" my-tests

The environment variable, specified by -e is important. Otherwise PHPUnit will run all of the unit tests it can find in your source tree.

There you go. Quick and easy Magium test runs in Docker.

AWS API Factory for the Magium Configuration Manager

This library provides an interface for the aws/aws-sdk-php library so you can use it with the Magium Configuration Manager. Often applications will have some kind of static configuration mechanism, such as XML files, JSON files, YAML files, or PHP files. There’s nothing necessarily wrong with that, but what it does is merge your deployment and configuration concerns. The Magium Configuration Manager (MCM) breaks that dependency so you can manage configuration separately from your deployment.

Setup

composer require magium/mcm-aws-factory

Once it is installed you need to initialize the Magium Configuration Manager (MCM) for your project using the magium-configuration commnand. You can find it in vendor/bin/magium-configuration or, if that doesn’t work you can run php vendor/magium/configuration-manager/bin/magium-configuration. For the purpose of this documentation we will simple call it magium-configuration.

Configuration

First, list all the configuration keys so you can see what they are.

$ magium-configuration magium:configuration:list-keys
Valid configuration keys
aws/credentials/region

aws/credentials/key

aws/credentials/secret

Then you need to set the settings:

$ magium-configuration set aws/general/region us-east-1
Set aws/general/region to us-east1 (context: default)
Don't forget to rebuild your configuration cache with magium:configuration:build

$ magium-configuration set aws/general/key xxxxxxxxxxxxxxxxxxx
Set aws/general/key to xxxxxxxxxxxxxxxxxxx (context: default)
Don't forget to rebuild your configuration cache with magium:configuration:build

$ magium-configuration set aws/general/secret xxxxxxxxxx
Set aws/general/secret to xxxxxxxxxxx (context: default)
Don't forget to rebuild your configuration cache with magium:configuration:build

Then you need to build the configuration:

$ magium-configuration build
Building context: default
Building context: production
Building context: development

Usage

Next up, in your application code run something like this:

$magiumFactory = new \Magium\Configuration\MagiumConfigurationFactory();
$awsFactory = new \Magium\AwsFactory\AwsFactory($magiumFactory->getConfiguration());

$ec2Client = $awsFactory->factory(\Aws\Ec2\Ec2Client::class);
$result = $ec2Client->describeSecurityGroups();
$groups = $result->get('SecurityGroups');

foreach ($groups as $count => $group) {
    echo sprintf("\nSecurity Group: %d\n", $count);
    foreach ($group as $name => $value) {
        if (is_string($value)) {
            echo sprintf("%s: %s\n", $name, $value);
        }
    }
}

Magium Active Directory Integration

Magium Active Directory Integration

For stupid-easy PHP integration with Azure Active Directory.

This is a simple library that uses the league/oauth2-client to provide OAuth2 based integration with Active Directory. Out of the box it is configured to work with Active Directory on Azure but, though I haven’t tested it, you can provide a different configuration object to the primary adapter and you should be able to authenticate against any Active Directory implementation as long as it has OAuth2 connectivity.

There are two purposes (well, three) for library.

  1. Provide sub-5 minute installation and integration times for any PHP-based application
  2. Provide a launching pad for other third-party integrations to Microsoft Azure Active Directory, such as Magento, Drupal, Oro, or whatever.
  3. (provide libraries that use other Magium libraries so people can see how awesome all the Magium stuff is)

First, watch the installation video on YouTube. It shows you how to create an application in Azure Active Directory.

Note Azure will not redirect from a secure URL (i.e. their login page) to an unsecure page (i.e. your page). No HTTPS to HTTP in other words. In yet other words, if you use Azure you will need to also use HTTPS. Though there are worse things in the world… like not using HTTPS.

Basic Usage

Anywhere in your application that requires authentication you can provide this code (properly architected, not cut and paste, in other words):

$ad = new \Magium\ActiveDirectory\ActiveDirectory(
    $configuration, // shown later
    $psr7CompatibleRequest
);

$entity = $ad->authenticate();

The authenticate() method will do 1 of 3 things.

  1. Check the session and see that the user is not logged in, forwarding that person to their Azure Active Directory login page
  2. Validate return data from Active Directory
  3. Simply return the Entity object if the person is already logged in.

If you want to log out all you do is:

$ad->forget();

Not that this only purges the AD entity from the session, it does not do any other session cleanup for your application.

Clearly this library is not intended to be your only means of session management, though, for simple applications, you could use it that way. Most likely you will want to take the data retrieved from AD and link it to a local account. The Entity class has 3 defined getters to help you do this mapping:

echo $entity->getName() . '<Br />'; // The user's name
echo $entity->getOid() . '<Br />'; //The user's AD object ID, useful for mapping to a local user obhect
echo $entity->getPreferredUsername() . '<Br />'; // The user's username, usually an email address.

Installation

composer require magium/active-directory

Done.

Configuration

This is a little more in-depth, but it shouldn’t be overly complex.

The base configuration is managed by the Magium Configuration Manager, out of the box. But, that said, the MCM has a really simple mechanism that allows you to not use the underlying plumbing. I believe that the underlying plumbing will eventually make application management easier, but I’m not going to force it on you.

Configuration using the Magium Configuration Manager

The configuration manager provides the means to manage and deploy settings at runtime in both a CLI and (eventually) a web-based interface. If you are using the configuration manager you need to get an instance of the configuration factory, which provides an instance of the manager, which provides the configuration object. The ActiveDirectory adapter requires that configuration object.


// Convert to PSR7 request object $request = \Zend\Psr7Bridge\Psr7ServerRequest::fromZend( new \Zend\Http\PhpEnvironment\Request() ); $factory = new \Magium\Configuration\MagiumConfigurationFactory(); $manager = $factory->getManager(); $configuration = $manager->getConfiguration(); $adapter = new \Magium\ActiveDirectory\ActiveDirectory($configuration, $request); $entity = $adapter->authenticate();

First, in your application root directory run vendor/bin/magium magium:configuration:list-keys. This is done after you have configured the MCM according to its instructions in the GitHub link. You will see output like this:

Valid configuration keys
magium/ad/client_id
        (You need to configure an application in Active Directory and enter its ID here)

magium/ad/client_secret
        (When you created an application in Active Directory you should have received a one-time use key.  Enter that here.)

You will need to provide those two values for the configuration:

vendor/bin/magium magium:configuration:set magium/ad/client_id '<my client id>'
Set magium/ad/client_id to <my client id> (context: default)
Don't forget to rebuild your configuration cache with magium:configuration:build

vendor/bin/magium magium:configuration:set magium/ad/client_secret '<my client secret>'
Set magium/ad/client_secret to <my client secret> (context: default)
Don't forget to rebuild your configuration cache with magium:configuration:build

vendor/bin/magium magium:configuration:build
Building context: default
Building context: production
Building context: development

And you should be good to go.

Configuration using PHP Arrays

Now, I know the MCM is new and you probably aren’t using it. That’s why I provided a way for you configure the adapter without using the full-blown MCM. You can use the Magium\Configuration\Config\Repository\ArrayConfigurationRepository class to provide a raw array that will be mapped to the two configuration settings magium/ad/client_id and magium/ad/client_secret


session_start(); $config = [ 'magium' => [ 'ad' => [ 'client_id' => '<my client id>', 'client_secret' => '<my client secret>' ] ] ]; $request = new \Zend\Http\PhpEnvironment\Request(); $ad = new \Magium\ActiveDirectory\ActiveDirectory( new \Magium\Configuration\Config\Repository\ArrayConfigurationRepository($config), Zend\Psr7Bridge\Psr7ServerRequest::fromZend(new \Zend\Http\PhpEnvironment\Request()) ); $entity = $ad->authenticate(); echo $entity->getName() . '<Br />'; echo $entity->getOid() . '<Br />'; echo $entity->getPreferredUsername() . '<Br />';

Configuration using YAML

Pretty much the same, but rather than using the ArrayConfigurationRepository you will use the YamlConfigurationRepository. It’s pretty similar:

$yaml = <<<YAML
magium:
    ad:
        client_id: value
        client_secret: value
YAML;

$obj = new YamlConfigurationRepository(trim($yaml));
$ad = new \Magium\ActiveDirectory\ActiveDirectory(
    $obj, $request
);

$entity = $ad->authenticate();

Configuration using JSON

Pretty much the same, but rather than using the YamlConfigurationRepository you will use the JsonConfigurationRepository. It’s pretty similar:

 $json = <<<JSON
        {
            "magium": {
                "ad": {
                    "client_id": "value"
                    "client_secret": "value"
                }
            }
        }
JSON;
        $obj = new JsonConfigurationRepository(trim($json));
$ad = new \Magium\ActiveDirectory\ActiveDirectory(
    $obj, $request
);

$entity = $ad->authenticate();

Configuration using INI Files

Pretty much the same, but rather than using the JsonConfigurationRepository you will use the IniConfigurationRepository. It’s pretty similar:

$ini = <<<INI
[magium]
ad[client_id] = value
ad[client_srcret] = value
INI;

$obj = new IniConfigurationRepository(trim($ini));
$ad = new \Magium\ActiveDirectory\ActiveDirectory(
    $obj, $request
);

$entity = $ad->authenticate();

How to handle configuration in PHP

Configuration management hasn’t changed much in PHP for, oh, the last 20 years or so. Basically, all of its life. Configuration file format has changed in that you can use XML, YAML, JSON, or PHP include files. But that is not configuration management. In other industries there are entire companies devoted to configuration management, but in PHP we are still largely relying on configuration files to provide our applications with variable configuration values.

I use the word “variable” loosely because oftentimes, out of the box, we need to do deployment changes to make configuration changes.

But what should we be doing?

In my humble opinion, the only configuration that should be deployed to a production environment is the configuration needed to get the rest of your configuration.

With that in mind I am announcing the pre-release availability of the Magium Configuration Manager, or MCM. While I have a ways to go, this is part of an idea I have had for at least 7 years for how I thought PHP development and deployment should be done, and it goes way beyond configuration management.

There are three primary purposes behind this library:

  1. Provide a standardized configuration panel/CLI so developers don’t need to do a code deployment to change a configuration value.
  2. Provide an inheritable interface where values can “bubble up” through child contexts
  3. Provide a unified (read merged) configuration interface for third parties that work in your application like magical unicorn dust.

The current version of the MCM is 0.8.22. Prior to a 1.x release I will be adding a web interface for humans to use (0.9) and a REST interface (0.10) for other applications to use (for example, a Node application could retrieve your PHP application’s SQS URL).

If you are using composer all you need to do to install the MCM is run

composer require magium/configuration-management

You will need several configuration files for your site. These are documented on the GitHub page. But for your specific application you will need a configuration file that describes your configuration needs. It will look something like this:

<configuration xmlns="http://www.magiumlib.com/Configuration">
    <section identifier="web">
        <group identifier="head">
            <element identifier="title" />
        </group>
    </section>
</configuration>

You will need to bootstrap the MCM for your site:

$factory = new \Magium\Configuration\MagiumConfigurationFactory();
$config = $factory->getManager()->getConfiguration(getenv('ENVIRONMENT'));

And get your configuration values

$siteTitle = $config->getPath('web/head/title');

I will have more to talk about as we go on, but for the time being you can use the following links for more information.

Two ways to Selenium test Twitter authentication using Magium

It’s been a while since I wrote the Twitter Selenium components in Magium and I forgot that I wrote two ways of authenticating. In case you don’t know how to install it: here you go

composer require magium/twitter

Directly authenticating against Twitter

        $this->getAction(AuthenticateTwitter::ACTION)->execute();

Site authentication using Twitter

        $action = $this->getAction(SignInWithTwitter::ACTION);

Two actions; two scenarios.

One thing you can do to make your Magento deployments more testable

Well, it actually two things.

There are two main problems that I’ve run into when working with Magium and browser testing. The first one is timing, particularly with Ajax requests. Ajax is somewhat hard to test because something may or may not happen and you have to create a lot of WebDriver wait() statements to manage it (Do not use sleep(), use wait() (That advice is free)).

The second is selecting things. I generally standardize on Xpath, particularly in Magium itself. I do this because it gives me a tremendous amount of control over getting the exact element I need. CSS selectors are usually good enough, but they can’t always get the element that you need. Xpath usually does.

But it does so at the expense of verbocity. This is the Xpath that I use to extract the product price from a product page

(//form[@id="product_addtocart_form"]/descendant::span[contains(concat(" ",normalize-space(@class)," ")," regular-price ")]/span[contains(concat(" ",normalize-space(@class)," ")," price ")])[1]

That is because the HTML on the Magento 1.9 CE site for the price is this (significantly redacted):

<form action="http://magento19.loc/checkout/cart/add/uenc/aHR0cDovL21hZ2VudG8xOS5sb2Mvd29tZW4vdG9wcy1ibG91c2VzL2VsaXphYmV0aC1rbml0LXRvcC00OTMuaHRtbD9fX19TSUQ9VQ,,/product/421/form_key/iMf3OyZB7cnrAleQ/" method="post" id="product_addtocart_form">
   <input name="form_key" type="hidden" value="iMf3OyZB7cnrAleQ">
   <div class="no-display">
      <input type="hidden" name="product" value="421">
      <input type="hidden" name="related_product" id="related-products-field" value="">
   </div>
   <div class="product-shop">
   <div class="product-name">
      <span class="h1">Elizabeth Knit Top</span>
   </div>
   <div class="price-info">
      <div class="price-box">
         <span class="regular-price" id="product-price-421">
                     <span class="price">$210.00</span>                                    
                </span>
      </div>
   </div>
</form>

You don’t see it here but there is also the possibility that several other prices are displayed in that form as well.

To build the Xpath I first need to find the closest, predictable element. Ideally that would be the element I need, itself, but having something close, such as a parent container, is often enough.

You might be thinking that I have that in this snippet. If you look at the parent container for the price you see id="product-price-421". However, that is only predictable for this one product. I cannot use it on another page. So the closest, predictable, element is the form element with id="product_addtocart_form". That must be the base.

Now I need to get to the price. “Simple!” you might say. “//span[@class="price"]“. Unfortunately, no. As I mentioned earlier, the unredacted form content has several price elements and this Xpath snippet would match all of them. So we can’t use it. But on top of that, it is a @class element. That means that there is no need for it to be unique and it may end up having other classes included for the purpose of styling. That is what causes our verbosity. That is why I have to end up using span[contains(concat(" ",normalize-space(@class)," ")," regular-price ")]. It pads the normalized @class string and checks to see if it contains regular-price (note the padding in there). That’s what selecting by class name requires.

And on top of that, the price is not predictable either. For that reason I need to wrap the entire selector in parenthesis and state I want result [1].

One Rule to Rule Them All

Use IDs to designate elements that have meaning.

I kid you not. If you made your HTML just a little more verbose by assigning IDs to elements that had meaning browser testing becomes a lot easier. That meaning could either be an action, such as a submit button, or a display element, such as the price.

This also helps with testing different types of browsers. Sometimes developers like to duplicate their effort by making different menus or elements for different screen resolutions. This ends up making testing somewhat difficult. But that said, if you tag the appropriate elements with meaningful names it becomes much easier. In the initial example this could be done by simply changing the ID.

<span class="regular-price" id="product-price-desktop">
                     <span class="price">$210.00</span>                                    
                </span>

That, alone, has just made testing this significantly easier and would change the selector

(//form[@id="product_addtocart_form"]/descendant::span[contains(concat(" ",normalize-space(@class)," ")," regular-price ")]/span[contains(concat(" ",normalize-space(@class)," ")," price ")])[1]

to

//span[@id="product-price-desktop"]/span[@class="price"]

(yes, I know I’m presuming nobody adds a CSS element to the price). But that alone makes element selection, the key part of Selenium testing, significantly easier.

Some Other Rules

Don’t Use Generated IDs

They are almost impossible to predict. In the original example this looked like <span class="regular-price" id="product-price-421">. That is naming worthy of Admiral Ozzel. It’s using the product primary key as part of the ID. That makes it impossible to predict with certainty.

Wrap Text In HTML Tags and Make It Specific

Generally speaking, you don’t want to select based off of text. Text is notoriously difficult to predict in testing scenarios and the fact the HTML doesn’t care about your whitespace makes it even more difficult. But if you MUST select on the text make sure that the important part is wrapped with no whitespace.

You might think that our previous example for the price is good:

<span class="price">$210.00</span>

But it actually isn’t. It denotes two pieces of distinct information: currency and price. A better approach would be something like this:

<span class="price" id="product-price-desktop">
    <span class="currency">$</span>
        <span class="value">210.00</span>
</span>

That approach provides a much more succinct means of selecting by important elements.

Select on @title or @data-* elements

If you can’t guarantee no whitespace select on attributes that can.

For example, if you want to click the XL button on a configurable product form that looks like this:

<ul id="configurable_swatch_size" class="configurable-swatch-list clearfix">
   <li class="option-s" id="option80">
      <a href="javascript:void(0)" name="s" id="swatch80" class="swatch-link swatch-link-180" title="S" style="height: 23px; min-width: 23px;">
      <span class="swatch-label" style="height: 21px; min-width: 21px; line-height: 21px;">
      S                                 </span>
      <span class="x">X</span>
      </a>
   </li>
   <li class="option-m" id="option79">
      <a href="javascript:void(0)" name="m" id="swatch79" class="swatch-link swatch-link-180" title="M" style="height: 23px; min-width: 23px;">
      <span class="swatch-label" style="height: 21px; min-width: 21px; line-height: 21px;">
      M                                 </span>
      <span class="x">X</span>
      </a>
   </li>
   <li class="option-l" id="option78">
      <a href="javascript:void(0)" name="l" id="swatch78" class="swatch-link swatch-link-180" title="L" style="height: 23px; min-width: 23px;">
      <span class="swatch-label" style="height: 21px; min-width: 21px; line-height: 21px;">
      L                                 </span>
      <span class="x">X</span>
      </a>
   </li>
   <li class="option-xs" id="option81">
      <a href="javascript:void(0)" name="xs" id="swatch81" class="swatch-link swatch-link-180" title="XS" style="height: 23px; min-width: 23px;">
      <span class="swatch-label" style="height: 21px; min-width: 21px; line-height: 21px;">
      XS                                 </span>
      <span class="x">X</span>
      </a>
   </li>
   <li class="option-xl" id="option77">
      <a href="javascript:void(0)" name="xl" id="swatch77" class="swatch-link swatch-link-180" title="XL" style="height: 23px; min-width: 23px;">
      <span class="swatch-label" style="height: 21px; min-width: 21px; line-height: 21px;">
      XL                                 </span>
      <span class="x">X</span>
      </a>
   </li>
</ul>

try the selector //a[@title="XL"] or, if you need more specificity, //ul[@id="configurable_swatch_size"]/descendant::a[@title="XL"]. It’s usually good to set the base of your selector to an ID since they are supposed to be unique on the page. You could also do //ul[@id="configurable_swatch_size"]//a[@title="XL"] but I’ve run into some scenarios (I don’t know why) where // does not work. So I just standardize on descendant::elementName.

Fin

Hopefully this gives you some hope that browser testing is doable. And hopefully you decide that Magium is the way to do it.

How can I change the directory the configuration is in?

It’s a simple problem. Perhaps you don’t want your abstract configurable element configurations in the root directory of your project. Prior to version 0.6.19 this may have been possible but the mechanism for doing it was quite cumbersome. With 0.6.19, the ConfigurationProviderInterface requires the configuration provider (StandardConfigurationProvider, in most cases) to configure the DiC. It’s kind of silly to have a class for configuration that doesn’t, itself, provide the mechanism for configuring DI.

It’s still a little wonky, but you can now easily change the configuration directory for your tests. Your project does not require, but should have, an abstract test case that extends either Magium\AbstractTestCase or Magium\Magento\AbstractMagentoTestCase. To change the configuration directory for the abstract configurable elements you need to overload the constructor like so:


abstract class AbstractTestCase extends \Magium\AbstractTestCase { public function __construct($name = null, array $data = [], $dataName = null, Initializer $initializer = null) { $initializer = new Initializer( null, null, new StandardConfigurationProvider( new ConfigurationReader(), new ClassConfigurationReader(realpath(__DIR__ . '/../configuration')), new EnvironmentConfigurationReader() ) ); parent::__construct($name, $data, $dataName, $initializer); } }

How to enable logging

The logging in Magium is fairly full featured. Almost all commands are passed through a logger. Out of the box logging is configured to use the Noop logger; i.e. no logging. However you can turn logging on quite easily by adding a file to the /configuration directory that has the appropriate logging configuration set

<?php

return [
    'instance'  => [
        Magium\Util\Log\Logger::class   => [
            'parameters'    => [
                'options'   => [
                    'writers' => [
                        [
                            'name' => \Zend\Log\Writer\Stream::class,
                            'options' => [
                                'stream' => 'f:/tmp/magium.log'
                            ]
                        ]
                    ]
                ]
            ]
        ]
    ]
];

/configuration/logging.php

How to change browser settings in a Selenium test

Magium tests are designed to be as cross browser compatible as possible. For that reason the Remote WebDriver component is used in all circumstances.

However, there might be some times when you need to test browser specific configurations. One example is setting the browser language. Each browser has different ways of doing it. So given that Magium is intended to be browser-independent, how do you make browser specific changes?

The key to this problem is the Initializer. It manages all of the configuration for an individual test and most of the time Magium will configure itself properly. But you can easily make changes in this process.

Start in your test class, or, preferably, a generic abstract test class that you have written for your site that is globally used for your tests (this allows for easy global changes). Override the setup() method, and prior to calling parent::setUp() provide the test class with a new Initializer.

class WebDriverArgumentsTest extends AbstractTestCase
{

    protected function setUp()
    {
        $this->initializer = new EspanolChromeInitializer();
        parent::setUp();
    }
}

It is in the code of the EspanolChromeInitialiser class where some of the magic occurs.

class EspanolChromeInitializer extends Initializer
{

    protected function getDefaultConfiguration()
    {
        $config = parent::getDefaultConfiguration();
        $capabilitities = $config['definition']['class']['Magium\WebDriver\WebDriverFactory']['create']['desired_capabilities']['default'];
        if ($capabilitities instanceof DesiredCapabilities) {
            $options = new ChromeOptions();
            $options->addArguments(['--lang=es']);
            $capabilitities->setCapability(ChromeOptions::CAPABILITY, $options);
        }
        return $config;
    }
}

This could also be done via DIC configuration but the DIC configuration is 100% global, so you would not be able to switch between one option and another. This approach allows you to change configuration on a per-test-class basis.

Validating Individual Shipping Methods

Out of the box Magium will select the first shipping method on the screen. But there might be some times when you need to validate a particular shipping method, or force an error to be thrown if a shipping method does not exist. ’tis now easy to do.


use Magium\Magento\AbstractMagentoTestCase; class ShippingTest extends AbstractMagentoTestCase { public function testSelectShipping() { $this->setPaymentMethod('CashOnDelivery'); $this->setShippingMethod('ByName'); $this->getShippingMethod()->setName('Fixed'); } }

Calling setName() will search the shipping HTML to find the first match that contains the specified text. So if you have a shipping method called “Priority Mail 2-Day Small Flat Rate Box“, entering “Priority Mail” will also match it.

If you want to validate that a shipping method is not available set the expected exception for the test as

$this->setExpectedException('Magium\Magento\Actions\Checkout\ShippingMethods\NoSuchShippingMethodException');