study-faraphel/M1-PE-Kafka
1
1
Fork 0
Code Issues Pull requests Projects 1 Releases Packages Wiki Activity Actions

Split the project into three modules for every actors #14

Merged
faraphel merged 3 commits from container-split into master 2024-07-05 00:39:36 +02:00
Conversation 0 Commits 3 Files changed 87 +85 -263
11 changed files with 85 additions and 263 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files
Showing only changes of commit 6434d27738 - Show all commits

26
README.md
View file

@ -23,26 +23,26 @@ Alternatively, you can build and run the project manually by using
[gradle](https://gradle.org/) and launching the application with [docker-compose](https://docs.docker.com/compose/). [gradle](https://gradle.org/) and launching the application with [docker-compose](https://docs.docker.com/compose/).
```shell ```shell
gradlew build # build the modules
(cd ./applications/consumer/ && ./gradlew build)
(cd ./applications/converter/ && ./gradlew build)
(cd ./applications/producer/ && ./gradlew build)
# start the docker based on the docker-compose.yaml file
docker compose up docker compose up
``` ```
## Configuration ## Configuration
If you wish, you can modify the configuration of the `docker-compose.yaml` file to fit your needs. The configuration can be modified in the [docker-compose.yaml](./docker-compose.yaml) file to fit your needs.
The container `application` can be easily modified with the following environment variables : You can find a list for each module of the project containing their individual configuration :
- [producer](./applications/producer/README.md)
| Name | Required | Format | Default | Description | - [consumer](./applications/consumer/README.md)
|------------------------------|----------|-----------------------------------------------------|--------------------------------------------------------------------|---------------------------------------------------------------| - [converter](./applications/converter/README.md)
| KAFKA_BOOTSTRAP_SERVERS | true | \<ip>[:port] | / | The Kafka server address |
| TOPIC_TEMPERATURE_CELSIUS | false | string of alphanumeric characters, ".", "_" and "-" | temperature-celsius | The name of the Kafka topic for the temperature in Celsius |
| TOPIC_TEMPERATURE_FAHRENHEIT | false | string of alphanumeric characters, ".", "_" and "-" | temperature-fahrenheit | The name of the Kafka topic for the temperature in Fahrenheit |
| TEMPERATURE_LOCATION | true | \<latitude>, \<longitude> | 49.9, 2.3 ([Amiens, France](https://fr.wikipedia.org/wiki/Amiens)) | The coordinates where to get the temperatures from |
## Expectation ## Expectation
The `application` container shall print the current temperature at the selected place in Fahrenheit every minute. The `application` container shall print the current temperature at the selected place in Fahrenheit every minute.
You can also access this value with the REST api at the `http://localhost:8080/temperature` endpoint. You can also access this value with the REST api at the `http://localhost:8080/v1/temperature` endpoint.
## References ## References
The project use the [Open-Meteo API](https://open-meteo.com/) to fetch the current temperature at the The project use the [Open-Meteo](https://open-meteo.com/) API to fetch the current temperature at the
given location. given location.

59
applications/common/README.md
View file

@ -1,58 +1,15 @@
# common # Module: Common
This project uses Quarkus, the Supersonic Subatomic Java Framework. This module contain common code that could be used between the three others modules of the application.
If you want to learn more about Quarkus, please visit its website: <https://quarkus.io/>. It implements some errors or some useful functionalities and structure to avoid redefining them in all the modules
and making debugging easier by only modifying a single file to fix the corresponding issue everywhere.
## Running the application in dev mode ## Build
You can run your application in dev mode that enables live coding using: This project is built automatically when building the others modules.
```shell script However, you can still build it manually with the following command :
./gradlew quarkusDev ```shell
```
> **_NOTE:_** Quarkus now ships with a Dev UI, which is available in dev mode only at <http://localhost:8080/q/dev/>.
## Packaging and running the application
The application can be packaged using:
```shell script
./gradlew build ./gradlew build
``` ```
It produces the `quarkus-run.jar` file in the `build/quarkus-app/` directory.
Be aware that it’s not an _über-jar_ as the dependencies are copied into the `build/quarkus-app/lib/` directory.
The application is now runnable using `java -jar build/quarkus-app/quarkus-run.jar`.
If you want to build an _über-jar_, execute the following command:
```shell script
./gradlew build -Dquarkus.package.jar.type=uber-jar
```
The application, packaged as an _über-jar_, is now runnable using `java -jar build/*-runner.jar`.
## Creating a native executable
You can create a native executable using:
```shell script
./gradlew build -Dquarkus.native.enabled=true
```
Or, if you don't have GraalVM installed, you can run the native executable build in a container using:
```shell script
./gradlew build -Dquarkus.native.enabled=true -Dquarkus.native.container-build=true
```
You can then execute your native executable with: `./build/common-1.0-SNAPSHOT-runner`
If you want to learn more about building native executables, please consult <https://quarkus.io/guides/gradle-tooling>.
## Related Guides
- Kotlin ([guide](https://quarkus.io/guides/kotlin)): Write your services in Kotlin

41
applications/common/src/main/kotlin/fr/faraphel/common/kafka/AdminUtils.kt
View file

@ -1,41 +0,0 @@
package fr.faraphel.common.kafka
import org.apache.kafka.clients.admin.Admin
import org.apache.kafka.clients.admin.AdminClientConfig
import org.apache.kafka.clients.admin.CreateTopicsResult
import org.apache.kafka.clients.admin.NewTopic
import java.util.*
/**
* A wrapper around KafkaAdminClient to simplify its configuration.
* @param server the kafka server address
* @param identifier the kafka identifier for the configuration
* @see Admin
*/
class AdminUtils(
private val server: String,
private val identifier: String = "admin"
) {
private val adminConfig = Properties().apply {
// set the identifier
this[AdminClientConfig.CLIENT_ID_CONFIG] = identifier
// set the server information
this[AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG] = server
}
private val admin = Admin.create(adminConfig)
/**
* Create the topics in the kafka server.
* @param topics the names of the topics to create.
* @return the result of the operation.
* @see Admin.createTopics
*/
fun createTopics(vararg topics: String): CreateTopicsResult {
// convert the topics name into the corresponding operation
val operations = topics.map { topic -> NewTopic(topic, 1, 1) }
// run the command
return this.admin.createTopics(operations)
}
}

1
applications/common/src/main/kotlin/fr/faraphel/common/temperature/Temperature.kt
View file

@ -19,4 +19,3 @@ class Temperature(
val asFahrenheit: Double val asFahrenheit: Double
get() = ((this.value - 273.15) * 1.8) + 32 get() = ((this.value - 273.15) * 1.8) + 32
} }

62
applications/consumer/README.md
View file

@ -1,58 +1,22 @@
# consumer # Module: Consumer
This project uses Quarkus, the Supersonic Subatomic Java Framework. This module contain the code responsible for receiving values, displaying them in the container output and hosting
the REST API where we can find the latest value received.
If you want to learn more about Quarkus, please visit its website: <https://quarkus.io/>. ## Build
## Running the application in dev mode This project is built automatically when using [docker-compose](https://docs.docker.com/compose/) at the root.
You can run your application in dev mode that enables live coding using: However, you can still build it manually with the following command :
```shell
```shell script
./gradlew quarkusDev
```
> **_NOTE:_** Quarkus now ships with a Dev UI, which is available in dev mode only at <http://localhost:8080/q/dev/>.
## Packaging and running the application
The application can be packaged using:
```shell script
./gradlew build ./gradlew build
``` ```
It produces the `quarkus-run.jar` file in the `build/quarkus-app/` directory. ## Configuration
Be aware that it’s not an _über-jar_ as the dependencies are copied into the `build/quarkus-app/lib/` directory.
The application is now runnable using `java -jar build/quarkus-app/quarkus-run.jar`. This module can be easily modified with the following environment variables :
If you want to build an _über-jar_, execute the following command: | Name | Required | Format | Default | Description |
|-------------------------|----------|-----------------------------------------------------|---------|---------------------------------------------------|
```shell script | KAFKA_BOOTSTRAP_SERVERS | true | \<ip>[:port] | / | The Kafka server address |
./gradlew build -Dquarkus.package.jar.type=uber-jar | TEMPERATURE_TOPIC | true | string of alphanumeric characters, ".", "_" and "-" | / | The Kafka topic were the temperature can be found |
```
The application, packaged as an _über-jar_, is now runnable using `java -jar build/*-runner.jar`.
## Creating a native executable
You can create a native executable using:
```shell script
./gradlew build -Dquarkus.native.enabled=true
```
Or, if you don't have GraalVM installed, you can run the native executable build in a container using:
```shell script
./gradlew build -Dquarkus.native.enabled=true -Dquarkus.native.container-build=true
```
You can then execute your native executable with: `./build/consumer-1.0-SNAPSHOT-runner`
If you want to learn more about building native executables, please consult <https://quarkus.io/guides/gradle-tooling>.
## Related Guides
- Kotlin ([guide](https://quarkus.io/guides/kotlin)): Write your services in Kotlin

1
applications/consumer/build.gradle.kts
View file

@ -27,6 +27,7 @@ dependencies {
// libraries // libraries
implementation("io.quarkus:quarkus-kafka-client") implementation("io.quarkus:quarkus-kafka-client")
implementation("io.quarkus:quarkus-kafka-streams") implementation("io.quarkus:quarkus-kafka-streams")
implementation("com.google.code.gson:gson:2.8.9")
} }
group = "fr.faraphel" group = "fr.faraphel"

9
applications/consumer/src/main/kotlin/fr/faraphel/consumer/Main.kt
View file

@ -24,12 +24,13 @@ class Main : QuarkusApplication {
server=kafkaServer, server=kafkaServer,
topic=topicTemperature topic=topicTemperature
) { message -> ) { message ->
// format the time of the message to a proper string // clean the values in the record
val time = timeFormatter.format(Instant.ofEpochMilli(message.timestamp())) val messageInstant = Instant.ofEpochMilli(message.timestamp())
val messageContent = message.value()
// print the value // print the value
println("[${time}] ${message.value()}°F") println("[${timeFormatter.format(messageInstant)}] ${messageContent}°F")
// update the value for the API // update the value for the API
TemperatureEndpoint.setTemperature(message.value()) TemperatureEndpoint.updateData(messageContent, messageInstant)
} }
// indefinitely consume new values // indefinitely consume new values

2
applications/consumer/src/main/kotlin/fr/faraphel/consumer/rest/PingEndpoint.kt
View file

@ -9,7 +9,7 @@ import jakarta.ws.rs.Path
* Always answer "Pong!" * Always answer "Pong!"
* Can be used to test if the API can be reached * Can be used to test if the API can be reached
*/ */
@Path("ping") @Path("/v1/ping")
class PingEndpoint { class PingEndpoint {
/** /**
* Handler for a GET request on this endpoint * Handler for a GET request on this endpoint

21
applications/consumer/src/main/kotlin/fr/faraphel/consumer/rest/TemperatureEndpoint.kt
View file

@ -1,23 +1,31 @@
package fr.faraphel.consumer.rest package fr.faraphel.consumer.rest
import com.google.gson.Gson
import jakarta.ws.rs.GET import jakarta.ws.rs.GET
import jakarta.ws.rs.Path import jakarta.ws.rs.Path
import jakarta.ws.rs.Produces
import jakarta.ws.rs.core.MediaType
import java.time.Instant
/** /**
* This API endpoint return the latest temperature measured (in Fahrenheit) * This API endpoint return the latest temperature measured (in Fahrenheit)
*/ */
@Path("temperature") @Path("/v1/temperature")
class TemperatureEndpoint { class TemperatureEndpoint {
companion object { companion object {
val jsonParser = Gson() ///< the json parser
private var temperature: Double? = null ///< the latest temperature value private var temperature: Double? = null ///< the latest temperature value
private var time: Instant? = null /// < the time of the latest value
/** /**
* Setter to update the latest temperature value * Setter to update the latest temperature value
* @param temperature the new temperature value * @param temperature the new temperature value
*/ */
fun setTemperature(temperature: Double) { fun updateData(temperature: Double, time: Instant) {
this.temperature = temperature this.temperature = temperature
this.time = time
} }
} }
@ -26,7 +34,10 @@ class TemperatureEndpoint {
* @return the latest temperature measured * @return the latest temperature measured
*/ */
@GET @GET
fun get(): Double? { @Produces(MediaType.APPLICATION_JSON)
return temperature fun get(): String = jsonParser.toJson(mapOf(
} "value" to temperature,
"time" to time?.toEpochMilli(),
"unit" to "Fahrenheit"
))
} }

63
applications/converter/README.md
View file

@ -1,58 +1,23 @@
# converter # Module: Converter
This project uses Quarkus, the Supersonic Subatomic Java Framework. This module contain the code responsible for receiving values from a topic, converting them from
Celsius temperature to Fahrenheit and send it back into another topic.
If you want to learn more about Quarkus, please visit its website: <https://quarkus.io/>. ## Build
## Running the application in dev mode This project is built automatically when using [docker-compose](https://docs.docker.com/compose/) at the root.
You can run your application in dev mode that enables live coding using: However, you can still build it manually with the following command :
```shell
```shell script
./gradlew quarkusDev
```
> **_NOTE:_** Quarkus now ships with a Dev UI, which is available in dev mode only at <http://localhost:8080/q/dev/>.
## Packaging and running the application
The application can be packaged using:
```shell script
./gradlew build ./gradlew build
``` ```
It produces the `quarkus-run.jar` file in the `build/quarkus-app/` directory. ## Configuration
Be aware that it’s not an _über-jar_ as the dependencies are copied into the `build/quarkus-app/lib/` directory.
The application is now runnable using `java -jar build/quarkus-app/quarkus-run.jar`. This module can be easily modified with the following environment variables :
If you want to build an _über-jar_, execute the following command: | Name | Required | Format | Default | Description |
|------------------------------|----------|-----------------------------------------------------|---------|-----------------------------------------------------------------|
```shell script | KAFKA_BOOTSTRAP_SERVERS | true | \<ip>[:port] | / | The Kafka server address |
./gradlew build -Dquarkus.package.jar.type=uber-jar | TEMPERATURE_CELSIUS_TOPIC | true | string of alphanumeric characters, ".", "_" and "-" | / | The Kafka topic were the temperature in celsius can be found |
``` | TEMPERATURE_FAHRENHEIT_TOPIC | true | string of alphanumeric characters, ".", "_" and "-" | / | The Kafka topic were the temperature in fahrenheit can be found |
The application, packaged as an _über-jar_, is now runnable using `java -jar build/*-runner.jar`.
## Creating a native executable
You can create a native executable using:
```shell script
./gradlew build -Dquarkus.native.enabled=true
```
Or, if you don't have GraalVM installed, you can run the native executable build in a container using:
```shell script
./gradlew build -Dquarkus.native.enabled=true -Dquarkus.native.container-build=true
```
You can then execute your native executable with: `./build/converter-1.0-SNAPSHOT-runner`
If you want to learn more about building native executables, please consult <https://quarkus.io/guides/gradle-tooling>.
## Related Guides
- Kotlin ([guide](https://quarkus.io/guides/kotlin)): Write your services in Kotlin

63
applications/producer/README.md
View file

@ -1,58 +1,23 @@
# producer # Module: Converter
This project uses Quarkus, the Supersonic Subatomic Java Framework. This module contain the code responsible for getting the current temperature at a given location thanks to the
[Open-Meteo](https://open-meteo.com/) API, and sending it into a topic.
If you want to learn more about Quarkus, please visit its website: <https://quarkus.io/>. ## Build
## Running the application in dev mode This project is built automatically when using [docker-compose](https://docs.docker.com/compose/) at the root.
You can run your application in dev mode that enables live coding using: However, you can still build it manually with the following command :
```shell
```shell script
./gradlew quarkusDev
```
> **_NOTE:_** Quarkus now ships with a Dev UI, which is available in dev mode only at <http://localhost:8080/q/dev/>.
## Packaging and running the application
The application can be packaged using:
```shell script
./gradlew build ./gradlew build
``` ```
It produces the `quarkus-run.jar` file in the `build/quarkus-app/` directory. ## Configuration
Be aware that it’s not an _über-jar_ as the dependencies are copied into the `build/quarkus-app/lib/` directory.
The application is now runnable using `java -jar build/quarkus-app/quarkus-run.jar`. This module can be easily modified with the following environment variables :
If you want to build an _über-jar_, execute the following command: | Name | Required | Format | Default | Description |
|-------------------------|----------|-----------------------------------------------------|---------|--------------------------------------------------------------|
```shell script | KAFKA_BOOTSTRAP_SERVERS | true | \<ip>[:port] | / | The Kafka server address |
./gradlew build -Dquarkus.package.jar.type=uber-jar | TEMPERATURE_TOPIC | true | string of alphanumeric characters, ".", "_" and "-" | / | The Kafka topic were the temperature in celsius can be found |
``` | TEMPERATURE_LOCATION | true | \<latitude>, \<longitude> | / | The coordinates where to get the temperatures from |
The application, packaged as an _über-jar_, is now runnable using `java -jar build/*-runner.jar`.
## Creating a native executable
You can create a native executable using:
```shell script
./gradlew build -Dquarkus.native.enabled=true
```
Or, if you don't have GraalVM installed, you can run the native executable build in a container using:
```shell script
./gradlew build -Dquarkus.native.enabled=true -Dquarkus.native.container-build=true
```
You can then execute your native executable with: `./build/producer-1.0-SNAPSHOT-runner`
If you want to learn more about building native executables, please consult <https://quarkus.io/guides/gradle-tooling>.
## Related Guides
- Kotlin ([guide](https://quarkus.io/guides/kotlin)): Write your services in Kotlin