New Feature: Import Subscribers from CSV file

More
8 years 8 months ago - 8 years 8 months ago #65399 by Christian Jull
Replied by Christian Jull on topic New Feature: Import Subscribers from CSV file
One HUGE failing in this import. It doesn't recognise non-Latin characters. I have a Finnish membership databse to import and it's absolutely filled with ä, ö, å characters, as well as other non-Latin characters like é ñ, etc.

The import just cuts the words at the point where one of these characters is. I tested by replacing the characters with standard Latin equivalents and the import worked fully. However, this is unacceptable. In Finnish, the difference between ä and a or ö and o in a word is like two completely different words in English.

Help please....

EDIT: And why are all subscribers set to 'pending', and why can't I change any of them?!!
Last edit: 8 years 8 months ago by Christian Jull. Reason: addition

Please Log in or Create an account to join the conversation.

  • Tuan Pham Ngoc
  • Topic Author
  • Offline
  • Administrator
  • Administrator
More
8 years 8 months ago - 8 years 8 months ago #65421 by Tuan Pham Ngoc
Replied by Tuan Pham Ngoc on topic New Feature: Import Subscribers from CSV file
Hi Christian

Could I please have your csv file so that I can check it myself? Maybe just give us a sample file which contains few records so that we can check it

For the reason it is pending, I think it happens because the published column you have the value set to 0 ? You can simply go to Membership Pro -> Subscriptions, publish/change status of these subscription records. That should work

Regards,

Tuan
Last edit: 8 years 8 months ago by Tuan Pham Ngoc.

Please Log in or Create an account to join the conversation.

More
8 years 8 months ago #65425 by Christian Jull
Replied by Christian Jull on topic New Feature: Import Subscribers from CSV file
I'll send you a sample of the CSV.

As for the published status, this screenshot clearly shows the problem.... The missing fields appear to vanish after the import.

Attachments:

Please Log in or Create an account to join the conversation.

More
8 years 8 months ago #65437 by Tony Partridge
Replied by Tony Partridge on topic New Feature: Import Subscribers from CSV file
Christian,

Can you check the database table collation? I have found this is usually down to this, on both the tables and the columns. The best collation to use is UTF8_GENERAL_CI

Thanks
Tony

Please Log in or Create an account to join the conversation.

More
8 years 8 months ago - 8 years 8 months ago #65438 by Christian Jull
Replied by Christian Jull on topic New Feature: Import Subscribers from CSV file
Thanks for the advisory, Tony. It is utf8_general_ci. This is a Finnish website and without it there would be character issues everywhere. It's pretty much the standard for all Scandinavian/Nordic websites. :)
Last edit: 8 years 8 months ago by Christian Jull.

Please Log in or Create an account to join the conversation.

More
8 years 8 months ago #65439 by Tony Partridge
Replied by Tony Partridge on topic New Feature: Import Subscribers from CSV file
Hi Christian,

Did you check the column collations too for the membership pro tables? I have seen it on quite a few occasions where the Joomla! tables are outfit_general_ci but some extensions are forced to latin on the table columns.

Thanks
Tony

Please Log in or Create an account to join the conversation.

More
8 years 8 months ago #65440 by Christian Jull
Replied by Christian Jull on topic New Feature: Import Subscribers from CSV file
Yep, they match - both before and after the import.

Please Log in or Create an account to join the conversation.

  • Tuan Pham Ngoc
  • Topic Author
  • Offline
  • Administrator
  • Administrator
More
8 years 8 months ago #65462 by Tuan Pham Ngoc
Replied by Tuan Pham Ngoc on topic New Feature: Import Subscribers from CSV file
Hi Christian

We tried to import the data on our local computer and it is imported properly. So our dev requested admin access to your site so that we can check it directly on your site to see what's wrong and get it sorted

Please provide us with the requested information

Tuan

Please Log in or Create an account to join the conversation.

More
8 years 8 months ago #65468 by Christian Jull
Replied by Christian Jull on topic New Feature: Import Subscribers from CSV file
I've sent site details. I tried disabling a few obvious things which may have been causing conflicts (e.g. Courseman), but that made no difference.

I'll need to know the specifics, as I'm working on a mirror test server currently and the changes need pushing to the live server.

Please Log in or Create an account to join the conversation.

  • Tuan Pham Ngoc
  • Topic Author
  • Offline
  • Administrator
  • Administrator
More
8 years 8 months ago #65472 by Tuan Pham Ngoc
Replied by Tuan Pham Ngoc on topic New Feature: Import Subscribers from CSV file
Hi Christian

We checked and found out the reason of the issue. The data in csv file is encoded in ANSI and could not be processed properly by the system.

We use notepad++ ( notepad-plus-plus.org ) "Convert To UTF-8 without BOM" to convert the text to UTF-8 encoding and the text can be imported properly.

My dev said that there were some text was changed but I think that's something we could not control it. So my suggestion is that you use that "Convert To UTF-8 without BOM" to convert the text to UTF-8 encoding (or using any software you want), then check the text and make sure it is correct. Then it will be imported properly by the system

The key requirement here is that the text of the csv file must be encoded using UTF-8 encoding.

Regards,

Tuan

Please Log in or Create an account to join the conversation.