DataWeave filter function: How to filter items in an Array

We would like to thank MuleSoft Ambassador Joshua Erney for their contribution to this developer tutorial.

The filter function was already used as an example for other DataWeave topics like lambdas and generics. However, in this tutorial, you’ll learn some examples or use cases in which you can filter an Array to only contain the items that pass a certain condition. We’ll show you how to write the prefix and infix notations, as well as the use of an explicit lambda and the dollar-sign syntax.

You can try all of these examples with the DataWeave Playground. To learn more about it, check out this tutorial.

Prerequisites

While not required to follow this tutorial, a good understanding of the basic DataWeave concepts would be preferred. You can check out these other tutorials if you feel a bit lost with some concepts:

• What is DataWeave? Part 1 - To understand MIME types, the script anatomy, and data types.
• What is DataWeave? Part 2 - To understand how to use the selectors to retrieve data.
• What is DataWeave? Part 3 - To understand variables, boolean operators, flow control, and named functions (prefix and infix notations).
• What is DataWeave? Part 4 - To understand what are lambdas (anonymous functions), higher-order functions, infix notation, and the dollar-sign syntax.
• What is DataWeave? Part 5 - To understand what is the syntax of the type parameters (or generics).

Syntax

To use filter, you will need to pass two arguments: an Array of any type and a function (or a condition) to filter the data with. The result will be of the same type as the input Array. For example, if you pass an Array of Numbers (Array<Number>), the result will also be an Array of Numbers, but it will contain just the filtered data according to the condition (or the second argument).

filter(Array<T>, ((T,Number) -> Boolean)): Array<T>

Prefix Notation

This function is not commonly used with the prefix notation because of its complexity. But here you can find an example to get the numbers that are more than 1:

Open in Playground

Infix Notation

This is the most used notation for this function because it makes it easier to read and understand.

Open in Playground

When using the infix notation, you don’t have to include all the parameters in the lambda if you’re not using them. However, you do have to include the previous parameters (to the left) whether you’re using them or not. You can only remove the parameters to the right.

In this case, since num is the first parameter, we can also use the following code:

1
2
3
4
%dw 2.0
output application/json
---
[1, 2, 3] filter ((num) -> num > 1)

Dollar-sign Syntax

You don’t always need to include the whole explicit lambda expression as the second argument. You can use the dollar-sign syntax to reference the two arguments that are passed to the lambda (i.e., num and numIndex). In this example, we are filtering all the numbers that are more than 2 and which index is more than 1. $ represents the value of the item (num), while $$ represents the index (numIndex).

Open in Playground

The same example with the explicit lambda can be written like this:

1
2
3
4
%dw 2.0
output application/json
---
[1, 2, 3] filter ((num, numIndex) -> num > 2 and numIndex > 1)

Filter an Array of Strings

Let’s start working with some examples. We’ll start with a simple one: how to filter an Array of Strings (Array<String>) to only include the Strings that contain the word Max.

Open in Playground

If you’re not familiar with the contains function, it can be used with two Strings to indicate whether the first String contains the second String (or substring). All the different ways to use contains are out of scope for this tutorial, but you can learn more about it in the documentation.

In this example, we used the infix notation and the dollar-sign syntax. We chose to write it this way because the code looks cleaner. If you wanted to use explicit lambda, while still using the infix notation, the same code would look like this:

1
2
3
4
5
6
7
8
%dw 2.0
output application/json
---
[
    "Hello World",
    "Hello Max",
    "Max the Mule"
] filter ((phrase) -> phrase contains "Max")

Filter a Simple Array of Objects

In this example, we have a JSON Array input payload containing objects. Each object has two properties: Name and Age. We will use the filter function to output a new Array of objects (Array<Object>) that only includes the people who are older than 20.

Open in Playground

For this example, we also used the infix notation and the dollar-sign syntax because there is no confusion on what the dollar-sign is referring to since it’s only two fields in each object. Let’s see what would happen if the input payload had more fields and values.

Filter a Complex Array of Objects

In this example, we have a more complex JSON Array input payload containing objects. Each object has three properties: Name, Age, and Phones. The Phones field contains another Array of objects. For our output, we want to only include the people who are older than 20 (as we previously did) and change the Phones field to PrimaryPhones. This new PrimaryPhones field should only include the phones with a Primary field equal to true.

If you’re not familiar with the map function, let’s just say that it’s a way to iterate through the Array. In this case, we need to iterate through the main Array to filter each phone from the Phones Array. How to use map is out of scope for this tutorial, but you can learn more about it in the documentation or this developer tutorial.

Dollar-sign syntax

Let’s first take a look at how this code would look like when we only use the dollar-sign syntax for every single lambda, including two filters and a map.

Open in Playground

As you can see, five different dollar signs are being referenced in this script: one for each filter function ($.Age and $.Primary), and three for the map function ($.Name, $.Age, $.Phones). You can get a bit lost in the code, especially if you’re not very familiar with this syntax or with DataWeave in general.

Explicit lambdas

Let’s now see how this same code would look like when using explicitly declared lambda functions with their parameters, instead of using the dollar-sign syntax.

Open in Playground

The first filter is still defined with the dollar-sign syntax because it can be understood that it’s referencing the complete payload. But we now defined the person and phone arguments in the map and the second filter. This lets you clarify to which object you’re referring to, instead of just calling all of them with $.

Next Steps

In this tutorial, you learned how to use the filter function with prefix and infix notations, as well as using the dollar-sign or explicit lambdas syntaxes. There is no best practice to use either notation or syntax. It depends on your preference. Some people may find the dollar-sign syntax easier to read, while others will prefer the explicit lambdas. The important thing is that you understand all of them.

Continue your development journey with the rest of the tutorials to become a master in DataWeave.