In the previous post, we started to show how to integrate Netgen Layouts with various content sources. We showed which groundwork needs to be done before doing more tangible things. In this post, we will cover several examples on how to build queries for dynamically fetching content from different content sources. By implementing queries in the code you enable Layouts users to configure in its UI which content will be fetched and shown. Remember, Layouts UI should enable daily changes to be efficient and usable even to non-developers.
Prerequisites
Before starting to work on your queries you should have done a few things covered in the last post:
- Understand main concepts of layouts and blocks and the architecture
- Layouts installed in some way
- Have a model in PHP for accessing your content
- Have a Symfony service with all specific integration code in one place
- ValueType for your content defined
- ValueConverter for your content created
- URLGenerator for your content created
- Twig templates for showing the item in Layouts UI and on the front side created
It looks like a long list but most of these steps are simple and straightforward.
More ways to fetch a content
Before we dig into queries let's first list two other ways of creating dynamic pages with Layouts.
-
Custom block
By implementing a custom block you can do whatever you like, including fetching and showing content. This might be useful for complex use cases where you have multidimensional data and special presentations like graphs, but when you want to show a simple collection of items it is better to go with queries. Benefits of implementing queries is less work, you can use the same query on different blocks, you can add items manually to the dynamic collections, etc. We will cover custom blocks in later posts. -
Your controller
Netgen Layouts is invoked only at the beginning of the Twig template rendered by your controller. We have a built-in “Twig block” in Layouts UI to render the snippets prepared in your Twig template’s blocks. Again, this is great for highly custom and complex use cases. The difference between custom blocks and this approach is that custom blocks can be reused on different pages independent from any controller.
Which approach to take really depends on your use case. If you have a one-dimensional list of objects coming from your local system (Database, AP...) or some external system (via REST, GraphQL, XML...) the preferred is another way — to create custom query types.
About QueryTypes
In Layouts, each block is defined to have a collection of items or not. This sets apart simple blocks such as titles from content-showing blocks like grids, lists or galleries.
A collection is holding content items coming from other systems or backends. Collection can be manual or dynamic. When using dynamic collection, users can select from a list of configured query types. After selecting a query type user can choose options (or parameters) which are used to perform the fetch.
The code for fetching and specifying query parameters is encapsulated in one PHP class we call Query Type. So to make a custom query you need to develop a QueryType which implements the QueryTypeHandlerInterface interface. There is an extensive documentation on this which I will not repeat here. It boils down to:
- Use built-in Parameter Types to build the form for query options in buildParameters(). With this you enable users to configure the query in the UI.
- Build the actual fetch in getValues() and getCount(), hopefully using the service and the model already prepared as shown in the last post.
- Finally, register the QueryType as Symfony service
It's important to note that your QueryType should return list objects from your model. Layouts internally uses the ValueConverter we mentioned in the last post to access generic data about the item (name, ID, etc.) and provide the original model object in the block view templates.
Collection already has built-in parameters for offset and limit so you don’t need to add those. You just need to use the passed offset and limit in the getValues() function when performing the fetch.
Regarding built-in parameter types, they should be enough for most of the use cases for QueryType options. But, as always, you will need to make something custom. There is a detailed description in our documentation.
Example 1: RSS/Atom QueryType
In the last post we prepared the ValueType and ValueConverter for the RSS integration. Now we will show how to implement the QueryType which actually fetches from the feed by using the "dg/rss-php" library.
First we need to configure the QueryType so that Layouts knows about it:
netgen_layouts:
query_types:
rss:
name: 'RSS/Atom feed'Then we add the needed parameters. In this case it is just 2 parameters – one text line for the URL and the other, choice type, for specifying the type of feed as the library supports two kind of feeds (but not automatically):
public function buildParameters(ParameterBuilderInterface $builder): void {
$builder->add('url', ParameterType\TextLineType::class);
$builder->add('feed_type',
ParameterType\ChoiceType::class, ['required' => true, 'options' => [ 'RSS' => 'rss', 'Atom' => 'atom'],]
);
}The core part is using the values of parameters and the library functions to fetch the items making sure we return a list of objects that can be used by Layouts with applicable Value Converter (shown in the last post):
public function getValues(Query $query, int $offset = 0, ?int $limit = null): iterable {
if ($query->getParameter('url')->isEmpty())
return [];
$url = $query->getParameter('url')->getValue();
$items = [];
if ($query->getParameter('feed_type')->getValue() == 'rss') {
$rss = \Feed::loadRss($url);
foreach( $rss->item as $item) {
$items[] = new RSSItem($item, 'rss');
}
} elseif ($query->getParameter('feed_type')->getValue() == 'atom') {
$rss = \Feed::loadAtom($url);
foreach( $rss->entry as $item) {
$items[] = new RSSItem($item, 'atom');
}
}
return $items;
}To make the example less complex we ignore the offset and limit, but it should be used in real code. The getCount() function would look more or less the same, just returning the total number of items.
Finally, we need to declare the class as service:
app.collection.query_type.handler.rss_query_type:
class: AppBundle\Layouts\RSSQueryTypeHandler
tags:
- { name: netgen_layouts.query_type_handler, type: rss }And that is it. You should see the query type in the dropdown when configuring the collections for your grid and list blocks. Find an RSS or Atom feed URL and copy & paste into the URL parameter to give it a try.
As a last touch, don’t forget to specify the labels in your project’s nglayouts.[language_code].yml:
query.rss.url: 'Feed URL'
query.rss.feed_type: 'Feed Type'Example 2: Some cats for entertainment
Mario did a blog post recently on using HTTP Clients with eZ Platform. Let’s reuse most of the stuff from that blog post and make a Layouts QueryType for fetching data from The Cat API. For this purpose, we’ve extended the API wrapper service with one additional method for fetching the breeds. Also, one notable change is that values are now represented by a simple value object, the TheCatApiValue which has 2 variables: ID and name. So our service class is a bit more advanced:
final class Service {
private $configResolver;
private $httpClient;
private $messageFactory;
public function __construct(
ConfigResolverInterface $configResolver,
HttpClient $httpClient,
MessageFactory $messageFactory
) {
$this->configResolver = $configResolver;
$this->httpClient = $httpClient;
$this->messageFactory = $messageFactory;
}
public function getCategories(): array {
$request = $this->messageFactory->createRequest(
Request::METHOD_GET,
$this->configResolver->getParameter('cats_api.url_categories', 'app'),
['x-api-key' => $this->configResolver->getParameter('cats_api.token', 'app')]
);
return $this->sendRequest($request);
}
public function getBreeds(): array {
$request = $this->messageFactory->createRequest(
Request::METHOD_GET,
$this->configResolver->getParameter('cats_api.url_breeds', 'app'),
['x-api-key' => $this->configResolver->getParameter('cats_api.token', 'app')]
);
return $this->sendRequest($request);
}
private function sendRequest(RequestInterface $request): array {
try {
$response = $this->httpClient->sendRequest($request);
} catch (\Exception $exception) {
return [];
}
if ($response->getStatusCode() !== Response::HTTP_OK) {
return [];
}
$data = json_decode( (string)$response->getBody(), true );
$objects = [];
foreach ($data as $datum) {
$objects[] = new TheCatApiValue($datum['id'], $datum['name']);
}
return $objects;
}
}Now when we have our value object defined, we need to tell Layouts how to convert it. This is done by creating a ValueConverter service. In our case it will look like this:
class TheCatApiValueConverter implements ValueConverterInterface {
public function supports(object $object): bool {
return $object instanceof TheCatApiValue;
}
public function getValueType(object $object): string {
return 'the_cat_api_value_type';
}
public function getId(object $object) {
return $object->getId();
}
public function getRemoteId(object $object) {
return $object->getId();
}
public function getName(object $object): string {
return $object->getName();
}
public function getIsVisible(object $object): bool {
return true;
}
public function getObject(object $object): object {
return $object;
}
}Let’s proceed with implementing Layouts QueryType. We want to be able to select sources, either to fetch the categories or breed. For this we use the ParameterType\ChoiceType parameter. Inside getValues() and getCount() we will use API wrapper service to fetch the data from the selected source.
class TheCatApiQueryType implements QueryTypeHandlerInterface {
private const SOURCE_BREEDS = 'breeds';
private const SOURCE_CATEGORIES = 'categories';
private $service;
public function __construct(Service $service) {
$this->service = $service;
}
public function buildParameters(ParameterBuilderInterface $builder): void {
$sources = [
'Get the Cat Breeds' => self::SOURCE_BREEDS,
'List the Categories' => self::SOURCE_CATEGORIES,
];
$builder->add( 'source', ParameterType\ChoiceType::class, ['required' => true, 'options' => $sources]);
}
public function getValues(Query $query, int $offset = 0, ?int $limit = null): iterable {
if ($query->getParameter('source')->getValue() == self::SOURCE_BREEDS) {
return $this->service->getBreeds();
}
return $this->service->getCategories();
}
public function getCount(Query $query): int {
if ($query->getParameter('source')->getValue() == self::SOURCE_BREEDS) {
return count($this->service->getBreeds());
}
return count($this->service->getCategories());
}
public function isContextual(Query $query): bool {
return false;
}
}And finally, we need to implement a few item templates, configure the QueryType name and register it as a service similar to other examples.
Example 3: Using GraphQL with eZ Platform
Recently we played a bit with the GraphQL support for eZ Platform, a great effort by Bertrand Dunogier from eZ.
It offers a much easier remote usage of eZ Platform content than the standard REST API. GraphQL is gaining popularity in how to consume data from remote repositories and it actually fits very well with eZ content model as it supports creating queries to hierarchical content.
Although GraphQL is mostly used from the frontside and JavaScript, there are also a lot of server-side use cases. We found a nice library which generates the model based on introspection of the GraphQL endpoint php-graphql-oqm.
It comes down to running php bin/generate_schema_objects, give the ez GraphQL endpoint: "http://your.ez.domain/graphql" and it will generate the code in the “GraphQL\SchemaObject” namespace.
Then you can, for example, make a query implementing fetching content of “article” type like this:
$rootObject = new GraphQL\SchemaObject\RootQueryObject();
$node = $rootObject->selectNetgen()->selectArticles()->selectEdges()->selectNode();
$node->selectTitle();
$node->selectUrl();
$node->selectImage()->selectId();
$info = $node->selectInfo();
$info->selectId()->selectPublishedDate()->selectFormat()->selectTimestamp();
$client = new \GraphQL\Client( 'http://your.ez.domain/graphql');
$res = $client->runQuery($rootObject->getQuery());What we get is an array of items as associative arrays. We get the title, URL, image and published date. If you need more data you can specify it in the query. Excellent thing with this generated model is that you should have code completion. Similar to the RSS example, it is necessary to have a simple value object that Layouts can detect as we described in the last post:
class EzGraphQLItem {
private $id;
private $title;
private $link;
public function __construct($object) {
$this->id = $object->node->_info->id;
$this->title = $object->node->title;
$this->link = $object->node->_url;
}
public function getId() {
return $this->id;
}
public function getTitle() {
return $this->title;
}
public function getLink() {
return $this->link;
}
}Also, you need to make a ValueConverter and the URLGenerator as described in the last post. With this you can then return the list of value objects in the getValues() function of the QueryType:
foreach ($res->getData()->netgen->articles->edges as $edge) {
$items[] = new EzGraphQLItem($edge);
}Making the QueryType configurable can be also done by using GraphQL. You can use the same generated code to build a query to create choice parameters like this:
$rootObject = new GraphQL\SchemaObject\RootQueryObject();
$rootObject->selectRepository()->selectContentTypes()->selectId()->selectName();Or you can execute raw GraphQL Introspection to get the whole specs:
$query = <<<EOD
query IntrospectionQuery {
__schema {
queryType { name }
types {
kind
name
fields { name }
interfaces {
kind
name
}
}
}
}
EOD;
$client = new \GraphQL\Client();
$res = $client->request('POST', 'http://your.ez.domain/graphql', [
'json' => [ 'query' => $query, 'variables' => null, 'operationName' => null ]
]);
$json = json_decode($res->getBody()->getContents());Then drill into this result to get things like content types, object states, etc. to make choice parameters.
Once you have the parameters built you can use them to filter results, for instance, filtering by parent location:
$parentLocationId = $query->getParameter('parent')->getValue();
$rootObject = new GraphQL\SchemaObject\RootQueryObject();
$argQuery = new GraphQL\SchemaObject\DomainGroupNetgenArticlesArgumentsObject();
$argQuery->setQuery(
(new GraphQL\SchemaObject\ContentSearchQueryInputObject())->setParentLocationId( array($parentLocationId) )
);
$node = $rootObject->selectNetgen()->selectArticles($argQuery)->selectEdges()->selectNode();The easiest way to test the content and introspection queries is to use the built-in interactive GraphQL interface "http://your.ez.domain/graphiql" or use the ChromeiQL extension. You can also browse generated documentation easily.
More examples
In each integration we made so far you can find a few QueryTypes for reference and inspiration:
Sylius
Check the “Last products” QueryType and “Taxon products” QueryType in the repository.
eZ Platform
There is a main query type with a lot of parameters (parent location, content type, section, object states) so you can easily filter out content you want. There are also a couple of QueryTypes in additional bundles to check: for filtering based on tags, and for showing related content:
Contentful
A simple query type for fetching content from Contentful REST API with parameters such as space and content type.
Few considerations
Once a user selects a QueryType in the block configuration, the fetch gets executed and fills the collection. Native feature of Layouts provides the possibility to add manual items on desired positions (slots). This is more valuable than you might think. Editors usually want to have things update automatically but prefer that they can override when necessary.
Caching
Collections have no caching by default, so this is something that needs to be implemented on the service level, especially if it is fetching content from a remote system. For example, eZ Platform integration uses the Repository API which transparently does persistence caching.
In the Contentful integration particular Contentful entry is cached locally.
In general, use caching provided by the library, SDK, Http client or whatever you have. If there is nothing a simple TTL based caching using Symfony cache should be straightforward to do.
Contextual parameters
So far we discussed parameters that are static in terms of the page we render. For example, if we are making a layout for a specific category page we can have a parameter to select a location from where we want to fetch a collection of items.
But what about the use case when you are making a layout for all category pages? In that case, the location of the items in the CMS is different and specific to each category so we need to declare that the QueryType is contextual. A good example is the Sylius Taxon QueryType which has a parameter “use_current_taxon” that enables this:
public function isContextual(Query $query): bool {
return $query->getParameter('use_current_taxon')->getValue() === true;
}This information is then used to enable getting the taxon from the current request and using it in the query fetch. In that way, if you have a layout mapped to taxon pages, each taxon page will show its context specific content.
Conclusion
As you can see from the post, making queries in Netgen Layouts is straightforward Symfony development. If you did the groundwork as described in the previous post, implementing the QueryType for your content should have been simple.
In the next post, we will cover integrating Content Browser with your content source. Content Browser is an independent component that Layouts uses to enable users to manually pick content items. It allows not just creating manually collections of items, but also putting manually selected items inside of a dynamic collection using queries. That is a unique feature of Layouts that offers high flexibility in content placement and prioritisation.
Stay tuned for the next post!