Importing a large number of subscribers

  • James Brew
  • Topic Author
  • Offline
  • New Member
  • New Member
More
8 years 4 months ago #92004 by James Brew
Importing a large number of subscribers was created by James Brew
Tuan
I am trying to import subscribers using the CSV import template. I have proved my csv file works correctly when the file is small. However, I have a file of 288,000 subscribers to import. After about 3,600 the import stops with a white screen with no message. Why is this happening?
1. Is there any sort of restriction on how many users I can import? e.g. PHP processing limit?
2. Is it possible to turn off the email address checking during import?
Thank you
James

Please Log in or Create an account to join the conversation.

More
8 years 4 months ago #92031 by Tuan Pham Ngoc
Replied by Tuan Pham Ngoc on topic Importing a large number of subscribers
Hello James

That depends on your server settings (memory limit and max_execute time)

However, I believe none of the system could import that large number of records. You should split it into smaller files.

Maybe each time, you should import only 2000 records (I know, it would take time to split but I am afraid of that's only solution)

The email check you are talking is handled by Joomla (Joomla requires each user to have a valid email), so we could not skip that check

Tuan

Please Log in or Create an account to join the conversation.

  • James Brew
  • Topic Author
  • Offline
  • New Member
  • New Member
More
8 years 4 months ago #92084 by James Brew
Replied by James Brew on topic Importing a large number of subscribers
I split the import into files of 2,000 records each. Imports were fine up to a total of 20,000 records then I started getting 504 Gateway Time-out Nginx errors. Siteground increased the Nginx server timeout to 600 seconds and some php variables as it follows:
max_execution_time = 3600
max_input_time = 3600
max_input_vars = 12000
However, I am still getting 504 Gateway Time-out Nginx errors.Should any of these settings be increased further to enable import to function?
Regards
James

Please Log in or Create an account to join the conversation.

More
8 years 3 months ago #92112 by Tuan Pham Ngoc
Replied by Tuan Pham Ngoc on topic Importing a large number of subscribers
Hi James

I don't see memory limit was increased? Maybe you should try to reduce number of records imported each time, for example 100 records each time to see how it works?

Tuan

Please Log in or Create an account to join the conversation.