Set up an automatic weekly process that needs 4-8 Gb RAM

Project: myhelpa-15611

As per client request we want to avoid calls to a Maps API to get time and distance between an important set of coordinates. We could do that by a process that reads a map file (from UK) and create indexed data readable to calculate time and distance. That could be done by having a recurring task on the server that downloads the updated map file and index its data. On local tests we have calculated that this kind of process would require the usage of 5-6 Gb once time per week. What would be the options that we have to set up something suitable?

cc @amador.jesus

Hi @jorge.m, thanks for your request.

Heroku scheduler should satisfy your needs. You could opt to use a more powerful dyno and store this data in memory. Or write it to disk. But keep in mind that this data will be erased once the job’s finished, so you need to upload it somewhere.

@dmitrii.k we have a super dyno on this project, but I think the limit is 2.5 Gb dedicated if not mistaken, can you double check?

Storage wise, can we upload the files to S3. Is there any file size limit we should know?

That’s correct, @jorge.m, you have a Performance-M dyno with 2.5GB RAM. However, Heroku Scheduler uses separate dynos, and you may opt to use any dyno you want, up to Performance-L with 14GB RAM. This may incur additional charges for the time it takes to run the job, depending on your subscription tier.

S3 single file size limit is 5TB, so you should be fine.

Not sure I’ve understood the minimum requirements. With Performance-M dyno, will we be able to run the process (5-6 Gb RAM) via Heroku scheduler?

If we decide to go for Performance-L, how much will be the extra cost?

Client subscription is Advance. Can we have a rough idea of the costs that the process might incur? b

Thanks in advance and apologies for the blast of questions :slight_smile:

@jorge.m,

For 5GB-6GB RAM you will have to go to Performance-L. But not for the entire app process, rather for this specific task once a week. So your app can continue running on Performance-M if that dyno satisfies the needs of the app; whilst your weekly process will run in a separate Performance-L dyno.

I think the best person to ask about the extra cost (if any) would be @anand, as this is a rather custom configuration.

You are always welcome! We’re here to support you, so please post any time! :raised_hands:

All clear. Thanks for your clarification @dmitrii.k . Looking forward to hearing from @anand regarding the costs of having the weekly process on a Performance-L dyno :slight_smile:

1 Like

@anand can we provide an estimate to the client? This feature is planned to be delivered within the next couple of weeks. Thanks in advance.