Forum

csv file i can down...
 
Notifications
Clear all

csv file i can download but cannot retrieve from PQ

7 Posts
3 Users
0 Reactions
73 Views
(@brunomelki)
Posts: 46
Trusted Member
Topic starter
 

Hello. 

I succeed to dowload the following file ( https://data.gov.il/dataset/f54e79b2-3e6b-4b65-a857-f93e47997d9c/resource/8a21d39d-91e3-40db-aca1-f73f7ab1df69/download/corona_city_table_ver_002.csv) without any issue when I click on the link in the site ( https://data.gov.il/dataset/covid-19/resource/8a21d39d-91e3-40db-aca1-f73f7ab1df69)

When I try to get the data from a New Query --> Other source --> Web, I cannot get anything.

What am I doing wrong ?

 
Posted : 21/10/2020 5:11 pm
(@mynda)
Posts: 4761
Member Admin
 

Hi Bruno,

Power Query can get data from a web page, but that link downloads a CSV file, it doesn't have the data stored in the web page. You need to download the file, then import it from the folder you downloaded it to with Power Query using From Text/CSV.

Mynda

 
Posted : 21/10/2020 8:58 pm
(@brunomelki)
Posts: 46
Trusted Member
Topic starter
 

Now it make sense since this is not the first time I see the same issue.

It means I cannot have an automatic update from Power Query from a CSV link. I must get the table itself since I see it is published frmo that specific site or to download it with a different tool (Python for example) prior to pull it to PQ as Text/CSV.

Any other possibilities without invoving Python ?

Bruno

 
Posted : 22/10/2020 5:29 am
(@brunomelki)
Posts: 46
Trusted Member
Topic starter
 

Mynda, I worked on that file with a friend and Python, this is a restriction by the publisher that want to avoid automation tools to overload their servers certainly. No possibility to download the file but to download it manualy every time.

Many thanks for your quick answer.

 
Posted : 22/10/2020 5:14 pm
(@mynda)
Posts: 4761
Member Admin
 

Ah, that's annoying.

 
Posted : 22/10/2020 8:17 pm
(@catalinb)
Posts: 1937
Member Admin
 

Hi,
You can automate the download using VBA and Internet Explorer, there is code in this forum to programatically click on a download button, most probably the download window can also be detected and the file saved somewhere.

https://www.myonlinetraininghub.com/web-scraping-filling-forms is an example to get you started.

 
Posted : 23/10/2020 7:25 am
(@brunomelki)
Posts: 46
Trusted Member
Topic starter
 

Many thanks for the link, will be helpfull since I will try to get a fully Excel automation. Actualy, I succeeded to get it thru Python via a "ckanapi" procedure. Something I didn't know before that.

Bruno

 
Posted : 03/11/2020 4:10 am
Share: