• Skip to main content
  • Skip to header right navigation
  • Skip to site footer

My Online Training Hub

Learn Dashboards, Excel, Power BI, Power Query, Power Pivot

  • Courses
  • Pricing
    • Free Courses
    • Power BI Course
    • Excel Power Query Course
    • Power Pivot and DAX Course
    • Excel Dashboard Course
    • Excel PivotTable Course – Quick Start
    • Advanced Excel Formulas Course
    • Excel Expert Advanced Excel Training
    • Excel Tables Course
    • Excel, Word, Outlook
    • Financial Modelling Course
    • Excel PivotTable Course
    • Excel for Customer Service Professionals
    • Multi-User Pricing
  • Resources
    • Free Downloads
    • Excel Functions Explained
    • Excel Formulas
    • Excel Add-ins
    • IF Function
      • Excel IF Statement Explained
      • Excel IF AND OR Functions
      • IF Formula Builder
    • Time & Dates in Excel
      • Excel Date & Time
      • Calculating Time in Excel
      • Excel Time Calculation Tricks
      • Excel Date and Time Formatting
    • Excel Keyboard Shortcuts
    • Excel Custom Number Format Guide
    • Pivot Tables Guide
    • VLOOKUP Guide
    • ALT Codes
    • Excel VBA & Macros
    • Excel User Forms
    • VBA String Functions
  • Members
    • Login
  • Blog
  • Excel Webinars
  • Excel Forum
    • Register as Forum Member

PQ loading from .csv corrupts data, a bug?|Power Query|Excel Forum|My Online Training Hub

You are here: Home / PQ loading from .csv corrupts data, a bug?|Power Query|Excel Forum|My Online Training Hub

Avatar
sp_LogInOut Log In sp_Registration Register
sp_Search Search
Advanced Search
Search
Forum Scope




Match



Forum Options



Minimum search word length is 3 characters - maximum search word length is 84 characters
sp_Search Search
sp_RankInfo
Lost password?
sp_CrumbsHome HomeExcel ForumPower QueryPQ loading from .csv corrupts data,…
sp_PrintTopic sp_TopicIcon
PQ loading from .csv corrupts data, a bug?
Avatar
Simon Smith
Member
Members
Level 0
Forum Posts: 26
Member Since:
May 19, 2021
sp_UserOfflineSmall Offline
1
December 15, 2022 - 7:33 am
sp_Permalink sp_Print

Hello everyone,

I've a weird issue. A .csv file that is large, but looks very normal when loaded into excel gets corrupted when power query picks it up into the 'transform data' stage. Column A is 'company name' from a database of suppliers to the government. Each row is a Cloud Hosting service that a supplier has put on the catalogue to make available to public sector customers to buy.

But when I load this to power query some (not all) of the data from columns EZ and beyond get displayed in Column A (and beyond) under the 'company name' - when they should be just in a single row of data going out to column GJ. I'll attach the .csv file that I generate from Scrapy taking the data off the government website. Copyright is fine for me to do this, it's all covered by the UK government Open Government Licence (OGL).

I have tried reformatting the .csv file to remove the alternate blank rows and also changed the asterisk (*) as a string separator, but it still makes this error. 

I'd be very grateful if someone has a solution for this. The Hosting1.csv file attached only has the first 100 rows of data (and alternate blanks) the live files I'm using have over 20,000 rows of data. The image file attached shows what I'm seeing when I load the data, the errors are visible in rows 16,23,26 and 33.

Many thanks

Simon (UK)0001.JPGImage Enlarger

sp_PlupAttachments Attachments
  • sp_PlupImage 0001.JPG (67 KB)
sp_AnswersTopicSeeAnswer See Answer
Avatar
Riny van Eekelen
Örnsköldsvik, Sweden
Moderator
Members


Trusted Members

Moderators

Power BI
Level 0
Forum Posts: 382
Member Since:
January 31, 2022
sp_UserOnlineSmall Online
2
December 15, 2022 - 4:36 pm
sp_Permalink sp_Print sp_EditHistory

It's not a bug. PQ simply allows you to treat quoted line breaks in different ways. The default is to apply them, but in this case that's not what you want. Press the cog wheel next to the Source step and change the Line breaks setting to ignore them. Then the last bit of Column156 (mgmt_dvc) plus all columns thereafter will NOT be displayed in a new row starting Column1.

Screenshot-2022-12-15-072330.pngImage Enlarger

sp_PlupAttachments Attachments
  • sp_PlupImage Screenshot-2022-12-15-072330.png (22 KB)
sp_AnswersTopicAnswer
Answers Post

The following users say thank you to Riny van Eekelen for this useful post:

Philip Treacy
Avatar
Simon Smith
Member
Members
Level 0
Forum Posts: 26
Member Since:
May 19, 2021
sp_UserOfflineSmall Offline
3
December 17, 2022 - 12:20 am
sp_Permalink sp_Print

Riny,

 

You are a genius.

 

A very helpful genius.

 

Thank you very much & happy christmas!

 

Simon

Avatar
Riny van Eekelen
Örnsköldsvik, Sweden
Moderator
Members


Trusted Members

Moderators

Power BI
Level 0
Forum Posts: 382
Member Since:
January 31, 2022
sp_UserOnlineSmall Online
4
December 17, 2022 - 1:59 am
sp_Permalink sp_Print

Glad I could help. Merry X-mas!

sp_Feed
Go to top
Forum Timezone: Australia/Brisbane
Most Users Ever Online: 170
Currently Online: mymalone, Riny van Eekelen, Ivan Kulubya, Vijay Vardhan
Guest(s) 85
Currently Browsing this Page:
1 Guest(s)
Top Posters:
SunnyKow: 1431
Anders Sehlstedt: 845
Velouria: 574
Purfleet: 412
Frans Visser: 346
David_Ng: 306
lea cohen: 213
A.Maurizio: 202
Aye Mu: 201
Jessica Stewart: 185
Newest Members:
Vicky Otosnika
Abhishek Singh
Kevin Sojourner
Kara Weiss
And Woox
Armani Quenga
moshood bello
annelies b
James1989
lucy gilmour
Forum Stats:
Groups: 3
Forums: 24
Topics: 6045
Posts: 26523

 

Member Stats:
Guest Posters: 49
Members: 31492
Moderators: 2
Admins: 4
Administrators: Mynda Treacy, Philip Treacy, Catalin Bombea, FT
Moderators: MOTH Support, Riny van Eekelen
© Simple:Press —sp_Information
  • Excel
  • Excel Charts
  • Excel Dashboard
  • Excel Formulas
  • Excel PivotTables
  • Excel Shortcuts
  • Excel VBA
  • General Tips
  • Online Training
  • Outlook
  • Power Apps
  • Power Automate
  • Power BI
  • Power Pivot
  • Power Query
 
  • About My Online Training Hub
  • Contact
  • Disclosure Statement
  • Frequently Asked Questions
  • Guarantee
  • Privacy Policy
  • Terms & Conditions
  • Testimonials
  • Become an Affiliate

Copyright © 2023 · My Online Training Hub · All Rights Reserved

Microsoft and the Microsoft Office logo are trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries. Product names, logos, brands, and other trademarks featured or referred to within this website are the property of their respective trademark holders.

Download A Free Copy of 100 Excel Tips & Tricks

excel tips and tricks ebook

We respect your privacy. We won’t spam you.

x