# How can i scale my temporal on EC2?

**URL:** <https://community.temporal.io/t/how-can-i-scale-my-temporal-on-ec2/6577>\
**Category:** Community Support\
**Tags:** general-impl\
**Created:** [November 27, 2022, 6:38pm UTC](https://community.temporal.io/t/how-can-i-scale-my-temporal-on-ec2/6577 "2022-11-27T18:38:19Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Obaid\_Ur\_Rehman](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/obaid_ur_rehman/32/2985_2.png) [@Obaid\_Ur\_Rehman](https://community.temporal.io/u/Obaid_Ur_Rehman)\
**Post date:** [November 27, 2022, 6:38pm UTC](https://community.temporal.io/t/how-can-i-scale-my-temporal-on-ec2/6577/1 "2022-11-27T18:38:19Z")

</div>

my Server is running fine on 1 EC2 instance and my Client UI is able to access it. (i am running auto-setup that comes default on temporal github)

and why does my workflow stays in the state of running, the behaviour is very random, some workflow gets completed in an instant some takes 2 minutes or sometimes never! even when they are ran parallelly. is this has to with EC2?

how do i scale on multiple instance, so in the case of one instance going down, other instance is able to continue the workflow. is it possible on EC2.  
thank you.

---

<div class="post-metadata">

**Author:** ![tihomir](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/tihomir/32/6580_2.png) [@tihomir](https://community.temporal.io/u/tihomir)\
**Post date:** [November 28, 2022, 3:12am UTC](https://community.temporal.io/t/how-can-i-scale-my-temporal-on-ec2/6577/2 "2022-11-28T03:12:22Z")

</div>

Do you have access to temporal server logs (if you are using auto-setup image, which btw is not really recommended for anything but testing, you would have all service logs together). Check for an frontend service errors maybe. Do you have server metrics enabled and are scraping them?

> why does my workflow stays in the state of running, the behavior is very random, some workflow gets completed in an instant some takes 2 minutes or sometimes never.

How do you deploy your worker(s), can you show workflow history one of of these workflows that take longer to complete / never complete?

> how do i scale on multiple instance, so in the case of one instance going down, other instance is able to continue

You would deploy multiple worker processes (your workers that you create and run your workflow code). Is this the question?

---

<div class="post-metadata">

**Author:** ![Obaid\_Ur\_Rehman](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/obaid_ur_rehman/32/2985_2.png) [@Obaid\_Ur\_Rehman](https://community.temporal.io/u/Obaid_Ur_Rehman)\
**Post date:** [December 1, 2022, 6:50am UTC](https://community.temporal.io/t/how-can-i-scale-my-temporal-on-ec2/6577/3 "2022-12-01T06:50:55Z")

</div>

OK so please ignore the question, since we decided we are not going forward with EC2.

we tried running your github link which you shared [github repo](https://github.com/tsurdilo/my-temporal-dockercompose)

and followed the steps

```auto
docker network create temporal-network
docker compose -f docker-compose-postgres.yml -f docker-compose-services.yml up

```

but the containers for **services and admin-tools** keep on restarting.

its throwing following error, (i just copied sample, let me know if you need full log)

temporal-worker | Unable to start server. Error: could not build arguments for function “[go.uber.org/fx](http://go.uber.org/fx)”.(\*App).constructCustomLogger.func2 (/go/pkg/mod/go.uber.org/fx@v1.17.1/app.go:415): failed to build fxevent.Logger: could not build arguments for function “[go.temporal.io/server/temporal](http://go.temporal.io/server/temporal)”.glob…func8 (/home/builder/temporal/temporal/fx.go:921): failed to build log.Logger: received non-nil error from function “[go.temporal.io/server/temporal](http://go.temporal.io/server/temporal)”.ServerOptionsProvider (/home/builder/temporal/temporal/fx.go:163): sql schema version compatibility check failed: unable to read DB schema version keyspace/database: temporal error: pq: relation “schema\_version” does not  
exist

temporal-worker | [Fx] PROVIDE \*pprof.PProfInitializerImpl \<= [go.temporal.io/server/common/pprof.NewInitializer()](http://go.temporal.io/server/common/pprof.NewInitializer())  
temporal-worker | [Fx] PROVIDE \*temporal.ServerImpl \<= [go.temporal.io/server/temporal.NewServerFxImpl()](http://go.temporal.io/server/temporal.NewServerFxImpl())  
temporal-worker | [Fx] PROVIDE temporal.Server \<= [go.temporal.io/server/temporal.glob..func9()](http://go.temporal.io/server/temporal.glob..func9())  
temporal-worker | [Fx] SUPPLY temporal.ServerOption

for all services.

please tell me if i am missing some steps? i thought this was supposed to run out of the box.  
thanks for your help.

question2, how do i run it, if i want to run it for only cassandra?

---

<div class="post-metadata">

**Author:** ![tihomir](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/tihomir/32/6580_2.png) [@tihomir](https://community.temporal.io/u/tihomir)\
**Post date:** [December 2, 2022, 6:53pm UTC](https://community.temporal.io/t/how-can-i-scale-my-temporal-on-ec2/6577/4 "2022-12-02T18:53:15Z")

</div>

Not able to reproduce the issue locally. Are you getting this after cleaning up docker (system, volume prune). For cassandra I assume you just want to connect to existing cassandra instance, not run it in container like done [here](https://github.com/temporalio/docker-compose/blob/main/docker-compose-cass.yml#L3) for example, can you confirm?

---

<div class="post-metadata">

**Author:** ![Obaid\_Ur\_Rehman](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/obaid_ur_rehman/32/2985_2.png) [@Obaid\_Ur\_Rehman](https://community.temporal.io/u/Obaid_Ur_Rehman)\
**Post date:** [December 5, 2022, 11:41am UTC](https://community.temporal.io/t/how-can-i-scale-my-temporal-on-ec2/6577/5 "2022-12-05T11:41:49Z")

</div>

i am testing it on _windows_ env. **to check if this could be an issue with my system i ran it on another system but it gave same errors**.

I cleaned up everything, recloned your repo and ran again with following commands

```auto
docker network create temporal-network
docker compose -f docker-compose-postgres.yml -f docker-compose-services.yml up

```

all 4 services + admin-tools just keeps restarting.

give following errors  
temporal-admin-tools | exec /etc/temporal/setup.sh: no such file or directory  
.  
.  
temporal-postgresql | 2022-12-05 11:15:07.609 UTC [47] ERROR: relation “schema\_version” does not exist at character 26  
temporal-postgresql | 2022-12-05 11:15:07.609 UTC [47] STATEMENT: SELECT curr\_version from schema\_version where version\_partition=0 and db\_name=

.  
.  
.  
temporal-frontend | Unable to start server. Error: could not build arguments for function “[go.uber.org/fx](http://go.uber.org/fx)”.(\*App).constructCustomLogger.func2 (/go/pkg/mod/go.uber.org/fx@v1.17.1/app.go:415): failed to  
build fxevent.Logger: could not build arguments for function “[go.temporal.io/server/temporal](http://go.temporal.io/server/temporal)”.glob…func8 (/home/builder/temporal/temporal/fx.go:921): failed to build log.Logger: received non-nil error from function “[go.temporal.io/server/temporal](http://go.temporal.io/server/temporal)”.ServerOptionsProvider (/home/builder/temporal/temporal/fx.go:163): sql schema version compatibility check failed: unable to read DB schema version keyspace/database: temporal error: pq: relation “schema\_version” does not exist  
.  
.  
.

temporal-frontend2 | [Fx] ERROR Failed to start: could not build arguments  
for function “[go.uber.org/fx](http://go.uber.org/fx)”.(\*App).constructCustomLogger.func2 (/go/pkg/mod/go.uber.org/fx@v1.17.1/app.go:415): failed to build fxevent.Logger: could not build arguments for function “[go.temporal.io/server/temporal](http://go.temporal.io/server/temporal)”.glob…func8  
(/home/builder/temporal/temporal/fx.go:921): failed to build log.Logger: received non-nil error from function “[go.temporal.io/server/temporal](http://go.temporal.io/server/temporal)”.ServerOptionsProvider (/home/builder/temporal/temporal/fx.go:163): sql schema version cere version\_partition=0 and db\_name=$1  
.  
.  
.

i am absolutely new to Temporal, if that will help you.  
am i missing a step? are there **prerequisite**? do i have to build schema or something?

---

<div class="post-metadata">

**Author:** ![tihomir](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/tihomir/32/6580_2.png) [@tihomir](https://community.temporal.io/u/tihomir)\
**Post date:** [December 5, 2022, 2:53pm UTC](https://community.temporal.io/t/how-can-i-scale-my-temporal-on-ec2/6577/6 "2022-12-05T14:53:06Z")

</div>

Thanks for the info, ok let’s try to figure out whats going on.

`temporal-admin-tools | exec /etc/temporal/setup.sh: no such file or directory`

Service definition for admin-tools sets a custom script as a volume [here](https://github.com/tsurdilo/my-temporal-dockercompose/blob/main/docker-compose-services.yml#L172). Wonder why that would be failing, this script is [in the repo](https://github.com/tsurdilo/my-temporal-dockercompose/blob/main/script/setup.sh).

Could you bash into your admin-tools container and see if its really not there or at some other location maybe?

```auto
docker ps
(find the container id for admin-tools image)
docker exec -it <container_id> bash
cd /etc/temporal/

```

and see if it was moved there. If not then we need to see why, maybe some sort of permissions issue?  
If the script is there check its permissions and try to run it manually in admin-tools container:

```auto
./etc/temporal/setup.sh
or 
bash /etc/temporal/setup.sh

```

also if you are not on latest of master branch please update your local repo, added some updates for nginx support recently so let’s make sure we are both on latest in repo

---

<div class="post-metadata">

**Author:** ![Obaid\_Ur\_Rehman](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/obaid_ur_rehman/32/2985_2.png) [@Obaid\_Ur\_Rehman](https://community.temporal.io/u/Obaid_Ur_Rehman)\
**Post date:** [December 5, 2022, 3:06pm UTC](https://community.temporal.io/t/how-can-i-scale-my-temporal-on-ec2/6577/7 "2022-12-05T15:06:06Z")

</div>

Hey

The admin-tools container keeps restarting.

```auto
Error response from daemon: Container a0d19fd05b3b2758699bbf616e3e9c83df383b460b252642de29f51f5c44ad90 is restarting, wait until the container is running

```

i havent moved any files, setup.sh is in /script folder.

do i have to make/create/move or rename any directory or files

---

<div class="post-metadata">

**Author:** ![tihomir](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/tihomir/32/6580_2.png) [@tihomir](https://community.temporal.io/u/tihomir)\
**Post date:** [December 5, 2022, 4:07pm UTC](https://community.temporal.io/t/how-can-i-scale-my-temporal-on-ec2/6577/8 "2022-12-05T16:07:11Z")

</div>

> [@Obaid\_Ur\_Rehman](#):
>
> The admin-tools container keeps restarting.

Can you try seeing its logs:

docker logs \<container\_id\>

---

<div class="post-metadata">

**Author:** ![tihomir](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/tihomir/32/6580_2.png) [@tihomir](https://community.temporal.io/u/tihomir)\
**Post date:** [December 5, 2022, 4:22pm UTC](https://community.temporal.io/t/how-can-i-scale-my-temporal-on-ec2/6577/9 "2022-12-05T16:22:34Z")

</div>

> [@Obaid\_Ur\_Rehman](#):
>
> do i have to make/create/move or rename any directory or files

I’m not sure how relative paths and volume work on windows honestly, specifically  
`./script/setup.sh` [here](https://github.com/tsurdilo/my-temporal-dockercompose/blob/main/docker-compose-services.yml#L172). Seeing stuff like [this](https://stackoverflow.com/questions/43181654/locating-data-volumes-in-docker-desktop-windows) and wonder if that can be some sort of path / permission problem or not.

---

<div class="post-metadata">

**Author:** ![lorenz](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/lorenz/32/3471_2.png) [@lorenz](https://community.temporal.io/u/lorenz)\
**Post date:** [March 14, 2023, 7:22pm UTC](https://community.temporal.io/t/how-can-i-scale-my-temporal-on-ec2/6577/10 "2023-03-14T19:22:12Z")

</div>

I had the same issue trying to get it working on a Windows machine. Converting the file my-temporal-dockercompose\script\setup.sh from DOS to Unix format fixed it for me. [see also](https://unix.stackexchange.com/questions/721844/linux-bash-shell-script-error-cannot-execute-required-file-not-found)  
You can for example use dos2unix setup.sh (in WSL) or Notepad++: Edit → EOL Conversion → Unix.