With this use case, we demonstrate how Quick can be used to create real-time customer profiles that can be populated with live data and queried by applications. In this example we build customer profiles for a music streaming service measuring customer metrics, making charts of the most listen albums, artists and tracks and creating recommendations based on the user's playlist.
Every time a customer listens to a song a listening event with the album, artist, track id and a timestamp is emitted to an Apache Kafka topic and then processed with Kafka Streams for customer profile creation.
You can find the corresponding code in our GitHub repository. The examples are based on the real world data set LFM-1b. The Kafka Streams application is written with our streams-bootstrap library.
For modeling and querying data in our example, we first define a schema with GraphQL. Query is one of the core operations in Quick and with getUserProfile we combine six metrics we want to query with the customer profile: The total listening events, the first and the last time a user listened to a song and charts with user’s most listened albums, artists and tracks. Later we will notice that the topics with those charts only contain ids and no names of the corresponding music data. Therefore, we resolve fields like id from topic topartists with artist names from topic artists in our GraphQL schema an call the corresponding type NamedArtistCount.
We are ready to process and query our data. We start by setting up our Quick instance. First, we initialize the Quick CLI, which requires a base URL and an API-Key. Second, we create a new gateway and apply our GraphQL schema.
Then, we create our main topics with the albums, artists and tracks data and the topic for storing all listening events. The command expects the topic name as well as the type or schema of key and value. Since we have complex values, we define a global GraphQL schema and apply it to the gateway. That way we will not need to specify a file, but use <name of the gateway>.<name of the type> from the global GraphQL schema for topic creation.
Next, we create the output topics needed for our Kafka Streams applications. Quick supports running dockerized applications: We can deploy those applications with the command 'quick app deploy...' that expects name, registry, image, tag (version) and arguments.
Finally, we want to add recommendations to our customer profiles. We can add to our global GraphQL Schema the example Query getArtistRecommendations that takes a few parameters, mandatory are only the userId and field, in our example we set field to ARTIST, but ALBUM or TRACK is also possible. The next four parameters come from the underlying recommendation algorithm SALSA and they are set by default.
Quick provides a custom directive @rest. Any type of REST service can be used in a Quick GraphQL schema. In our example, the result from the recommendation algorithm for a particular user id is return via REST as a list of artist ids. Since we want to recommend artist names, we resolve the ids from the REST service with names from the artists topic.