XDomPath vs Xpath or new query language which can handle ShadowDOM

5 min readJul 30, 2019

Xpath is very powerful query language which is widely used by lots of developers and (especially) QA automation engineers. Very often xpath becomes main engine for querying DOM nodes inside Selenium projects. It’s ability to walk up or down, to apply lots of different filters and functions makes it a perfect candidate for this role.

What have changed?

Web development technologies are very dynamic and changing unbelievably fast. Why? Because world of web services is growing and needs of users are growing as well. UIs become much and much more complicated and it makes a real headache for developers who have to take care about the design of their apps to be as much as possible easy to support and scale.

Web components technology is pretty new but already have made lots of noise. It gives ability to create native reusable html elements which can be plugged in declaratively. One of the most important technology of web components is ShadowDOM.

ShadowDOM — new feature of DOM API which provides an ability to encapsulate your DOM subtree. So as result you can design your new reusable html elements which can have own isolated DOM with isolated styles.

DOM with ShadowDOM

Let’s take a look at the simple example with ShadowDOM:

Hmm, isn’t it looks a bit different from the clear XML? Shadow root is like a new root of another DOM subtree? In this case we see that DOM tree is not a clear XML anymore.

Xpath and ShadowDOM?

ShadowDOM breaks XML structure of a DOM. How to access scoped Shadow tree nodes via xpath? What if I want to just find some input element with some particular class without sticking to the shadow subtree deepness?

Let’s try if we can find a span inside of ShadowDOM:

Ouch! Seems like Xpath cannot search deeply through the ShadowDOM. So is it suitable now for same tasks as before? If we need to work with our DOM as one global document of elements without care of ShadowDOM deepness we need an ability to query as before.

You can ask — is it safe? ShadowDOM was created for security reasons and we are going to reduce its encapsulation and use it as before. My vision that for automation purposes it is fine since the main goal is to find element (not to modify or remove). For developers purposes it also can suit, but to my opinion only in some very specific cases (it shouldn’t be main pattern of querying nodes in runtime code), but the idea should be the same — use it only to find elements.

What is the solution? XDomPath language

What query languages do we have for DOM tree? Specific only for DOM tree. Not sure if we can name something more than just CSS. Is CSS bad? No, but Xpath is much more powerful and it would be very useful if we can continue using these features in future.

This was my main purpose of creating new DOM query Xpath-based language — XDomPath.

You can read more tecnhical details about setup and installation by link to my library in GitHub: https://github.com/telenko/xdompath

XDomPath XML Model

XDomPath tries to treat DOM with ShadowDOM as simple XML (screenshot below)

So algorithm of mapping DOM to XML looks like this:

ShadowRoot node is always first children of it’s host
ShadowRoot’s host is it’s direct parent
Slotted elements are direct children of host (not slot child elements)
All other nodes relationships are not impacted

So how is it works? Let’s try a few queries

Example 1. Simple deep searching. XDomPath query “.//div”

We see that XDomPath doesn’t require us to deep inside of particular ShadowDOM scope manually. By following algorithm above it will process shadow root element as usual children of div (first and second) container.

Example 2. Searching for shadow roots. XDomPath query “.//shadow()”

Same as text() function we can use new one shadow() which returns shadowRoot NodeList.

Example 3. Searching for text nodes. XDomPath query “.//text()[.=’some-text’]”

Example 4. Slotted elements querying. XDomPath query “.//div/text()”

Since slotted element treats as direct children of host element this query will return single text node.

Example 5. Ensuring that element is located under the shadow root. XDomPath query “.//shadow()/p”

Example 6. Finding containers which have shadow root. XDomPath query “.//div[./shadow()]”

Programming API

XDomPath is a Javascript library which is available as npm dependency. Below is an example of including XDomPath into a code-base.

Basic XDomPath usage:

2. Using new features:

3. Planned feature of defining manual functions:

Selenium WebDriver Java annotation support

To add support of XDomPath for Selenium automation Java projects I’ve created a new annotation FindByQuery which extends regular FindBy with new query option — XDomPath.

You can find it by the link: https://github.com/telenko/find-by-query

Here is how it can be used inside of Java code-base: