• Skip to main content
  • Skip to header right navigation
  • Skip to site footer

My Online Training Hub

Learn Dashboards, Excel, Power BI, Power Query, Power Pivot

  • Courses
  • Pricing
    • Free Courses
    • Power BI Course
    • Excel Power Query Course
    • Power Pivot and DAX Course
    • Excel Dashboard Course
    • Excel PivotTable Course – Quick Start
    • Advanced Excel Formulas Course
    • Excel Expert Advanced Excel Training
    • Excel Tables Course
    • Excel, Word, Outlook
    • Financial Modelling Course
    • Excel PivotTable Course
    • Excel for Customer Service Professionals
    • Excel for Operations Management Course
    • Excel for Decision Making Under Uncertainty Course
    • Excel for Finance Course
    • Excel Analysis ToolPak Course
    • Multi-User Pricing
  • Resources
    • Free Downloads
    • Excel Functions Explained
    • Excel Formulas
    • Excel Add-ins
    • IF Function
      • Excel IF Statement Explained
      • Excel IF AND OR Functions
      • IF Formula Builder
    • Time & Dates in Excel
      • Excel Date & Time
      • Calculating Time in Excel
      • Excel Time Calculation Tricks
      • Excel Date and Time Formatting
    • Excel Keyboard Shortcuts
    • Excel Custom Number Format Guide
    • Pivot Tables Guide
    • VLOOKUP Guide
    • ALT Codes
    • Excel VBA & Macros
    • Excel User Forms
    • VBA String Functions
  • Members
    • Login
    • Password Reset
  • Blog
  • Excel Webinars
  • Excel Forum
    • Register as Forum Member

Power Query M messy noisy data file non adjacent data|Power Query|Excel Forum|My Online Training Hub

You are here: Home / Power Query M messy noisy data file non adjacent data|Power Query|Excel Forum|My Online Training Hub
Avatar
sp_LogInOut Log In sp_Registration Register
sp_Search Search
Advanced Search|Last Search Results
Search
Forum Scope




Match



Forum Options



Minimum search word length is 3 characters - maximum search word length is 84 characters
sp_Search Search
sp_RankInfo
Lost password?
sp_CrumbsHome HomeExcel ForumPower QueryPower Query M messy noisy data file…
sp_PrintTopic sp_TopicIcon
Power Query M messy noisy data file non adjacent data
Avatar
Robert Rafferty
Member
Members
Level 0
Forum Posts: 32
Member Since:
September 15, 2020
sp_UserOfflineSmall Offline
1
October 27, 2020 - 7:00 am
sp_Permalink sp_Print

Do have some insight or advice you can give me on how to transform this messy non adjacent data into a useable row and column structure?

I believe Power Query is made for this type of transformation, however, I am not sure of the best or standard approach using interface or M code after doing some testing and searching

This is one of 500 blocks, so I want to repeat after I build the query.

I could use Excel and VBA which I know exactly what steps and approach, although it seems PQ was designed to do this clean up and restructure.

I am thinking I should index or group each block somehow as a starting point, but this is really new territory

 

I did some fairly involved transformations before in my corporate life, but data set was simpler.

sp_AnswersTopicSeeAnswer See Answer
Avatar
Mynda Treacy
Admin
Level 10
Forum Posts: 4446
Member Since:
July 16, 2010
sp_UserOfflineSmall Offline
2
October 27, 2020 - 9:02 am
sp_Permalink sp_Print

Hi Robert,

You haven't provided an example of how you would like this data to look once it has been transformed, but assuming the data in the first 31 rows needs adding to columns in rows 32 to 43 then I would do this in two parts by referencing the data in two queries, one for the first 31 rows and one for the remainder. Then you can fix the first 31 rows and then join it to the data in rows 32 to 43.

Something that looks odd is that the data in the first 31 rows doesn't contain any actual information e.g Work Order Entered Date appears to contain the number 10, which obviously isn't a date. And looking at all of the data in the top half each field only contains a number, so you could possibly just remove the top section of data because it looks useless.

If that's the case, you can use Power Query to Remove the Top 29 rows, then remove the empty columns and you're done. 

Mynda

Avatar
Robert Rafferty
Member
Members
Level 0
Forum Posts: 32
Member Since:
September 15, 2020
sp_UserOfflineSmall Offline
3
October 28, 2020 - 5:32 am
sp_Permalink sp_Print

Thanks, I will follow up with end state example to make sure I am clearer.

In meantime, I will do suggested steps.

Regards,

Robb

Avatar
Robert Rafferty
Member
Members
Level 0
Forum Posts: 32
Member Since:
September 15, 2020
sp_UserOfflineSmall Offline
4
October 28, 2020 - 6:31 am
sp_Permalink sp_Print

Hi Mynda:

 

Please see complete tab, this is a better, more complete example.

Its pretty messy and mixed, but anything will help.  I need to repeat for 500 other groups, too, but custom function is for another day.

 

Robb

Avatar
Mynda Treacy
Admin
Level 10
Forum Posts: 4446
Member Since:
July 16, 2010
sp_UserOfflineSmall Offline
5
October 28, 2020 - 10:15 am
sp_Permalink sp_Print

Hi Robb,

There's no easy way to do this. In the attached file I've created 4 queries for the header data, one for each set, then appended them together into one table (headers).

The transaction data is then loaded in a separate table and you can either merge or append this to the headers. In the attached file I've done both.

Hope that helps.

Mynda

sp_AnswersTopicAnswer
Answers Post
Avatar
Robert Rafferty
Member
Members
Level 0
Forum Posts: 32
Member Since:
September 15, 2020
sp_UserOfflineSmall Offline
6
October 29, 2020 - 8:01 am
sp_Permalink sp_Print

Thanks, good to know its doable with these queries.

 

Thanks

Robb

sp_Feed
Go to top
Forum Timezone: Australia/Brisbane
Most Users Ever Online: 245
Currently Online: Richard Mander, Peter Hoffman
Guest(s) 10
Currently Browsing this Page:
1 Guest(s)
Top Posters:
SunnyKow: 1432
Anders Sehlstedt: 870
Purfleet: 412
Frans Visser: 346
David_Ng: 306
lea cohen: 219
A.Maurizio: 202
Jessica Stewart: 202
Aye Mu: 201
jaryszek: 183
Newest Members:
John Chisholm
vexokeb sdfg
John Jack
Malcolm Toy
Ray-Yu Yang
George Shihadeh
Naomi Rumble
Uwe von Gostomski
Jonathan Jones
drsven
Forum Stats:
Groups: 3
Forums: 24
Topics: 6212
Posts: 27236

 

Member Stats:
Guest Posters: 49
Members: 31888
Moderators: 3
Admins: 4
Administrators: Mynda Treacy, Philip Treacy, Catalin Bombea, FT
Moderators: MOTH Support, Velouria, Riny van Eekelen
© Simple:Press —sp_Information

Sidebar

Blog Categories

  • Excel
  • Excel Charts
  • Excel Dashboard
  • Excel Formulas
  • Excel PivotTables
  • Excel Shortcuts
  • Excel VBA
  • General Tips
  • Online Training
  • Outlook
  • Power Apps
  • Power Automate
  • Power BI
  • Power Pivot
  • Power Query
microsoft mvp logo
trustpilot excellent rating
Secured by Sucuri Badge
MyOnlineTrainingHub on YouTube Mynda Treacy on Linked In Mynda Treacy on Instagram Mynda Treacy on Twitter Mynda Treacy on Pinterest MyOnlineTrainingHub on Facebook
 

Company

  • About My Online Training Hub
  • Disclosure Statement
  • Frequently Asked Questions
  • Guarantee
  • Privacy Policy
  • Terms & Conditions
  • Testimonials
  • Become an Affiliate

Support

  • Contact
  • Forum
  • Helpdesk - For Technical Issues

Copyright © 2023 · My Online Training Hub · All Rights Reserved. Microsoft and the Microsoft Office logo are trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries. Product names, logos, brands, and other trademarks featured or referred to within this website are the property of their respective trademark holders.