I want to use the atomix framework using the java API in my application.
The application should be deployed and scaled via kubernetes. And every Pod should be "connected" with all Pods of the same kubernetes deployment.
I have seen in the documentation that there are different ways how to set up cluster discovery, so that each node of the cluster knows all members but no configuration seems to work for my scenario.
There is also a section about kubernetes deployment in the atomix manual but it seems that this is only useful for launching multiple atomix agents in a cluster and not for scaling a custom application which uses the Atomix API (please let me know if I get this wrong)
I don't have found any examples for such a setting even if it should be quite a common task to solve...
You can use DNS provided you configure a service specifically for this task. In k8s, every pod can be a member of an arbitrary number of services (because a service is just a load-balancer). So you could define a service for this purpose
---
apiVersion: v1
kind: Service
metadata:
labels:
- myLabel
name: service-clustering
spec:
clusterIP: None
publishNotReadyAddresses: true
ports:
- name: appName
port: 8080
protocol: TCP
targetPort: 8080
selector:
- matchLabel
type: ClusterIP
Important here is the publishNotReadyAddresses
, because you want to prevent a split-brain scenario during bootup before all pods passed their readiness checks. Afterwards you can just use DNS to discover the individual pods (using dnsjava
):
private Stream<String> getPodUris() {
return Optional.ofNullable(new Lookup("service-clustering", Type.SRV))
.stream()
.flatMap(Arrays::stream)
.filter(r -> r instanceof SRVRecord)
.map(r -> ((SRVRecord) r).getTarget().toString());
}
For dynamic scaling you'd need to repeat this query in a scheduled task every couple of seconds to inform Atomix about membership changes.