• Skip to main content
  • Skip to header right navigation
  • Skip to site footer

My Online Training Hub

Learn Dashboards, Excel, Power BI, Power Query, Power Pivot

  • Courses
  • Pricing
    • Free Courses
    • Power BI Course
    • Excel Power Query Course
    • Power Pivot and DAX Course
    • Excel Dashboard Course
    • Excel PivotTable Course – Quick Start
    • Advanced Excel Formulas Course
    • Excel Expert Advanced Excel Training
    • Excel Tables Course
    • Excel, Word, Outlook
    • Financial Modelling Course
    • Excel PivotTable Course
    • Excel for Customer Service Professionals
    • Excel for Operations Management Course
    • Excel for Decision Making Under Uncertainty Course
    • Excel for Finance Course
    • Excel Analysis ToolPak Course
    • Multi-User Pricing
  • Resources
    • Free Downloads
    • Excel Functions Explained
    • Excel Formulas
    • Excel Add-ins
    • IF Function
      • Excel IF Statement Explained
      • Excel IF AND OR Functions
      • IF Formula Builder
    • Time & Dates in Excel
      • Excel Date & Time
      • Calculating Time in Excel
      • Excel Time Calculation Tricks
      • Excel Date and Time Formatting
    • Excel Keyboard Shortcuts
    • Excel Custom Number Format Guide
    • Pivot Tables Guide
    • VLOOKUP Guide
    • ALT Codes
    • Excel VBA & Macros
    • Excel User Forms
    • VBA String Functions
  • Members
    • Login
    • Password Reset
  • Blog
  • Excel Webinars
  • Excel Forum
    • Register as Forum Member

Row data getting mixed up after expanding a merged query|Power Query|Excel Forum|My Online Training Hub

You are here: Home / Row data getting mixed up after expanding a merged query|Power Query|Excel Forum|My Online Training Hub
Avatar
sp_LogInOut Log In sp_Registration Register
sp_Search Search
Advanced Search|Last Search Results
Search
Forum Scope




Match



Forum Options



Minimum search word length is 3 characters - maximum search word length is 84 characters
sp_Search Search
sp_RankInfo
Lost password?
sp_CrumbsHome HomeExcel ForumPower QueryRow data getting mixed up after exp…
sp_PrintTopic sp_TopicIcon
Row data getting mixed up after expanding a merged query
Avatar
Mike Magill

Active Member
Members
Level 0
Forum Posts: 3
Member Since:
August 18, 2022
sp_UserOfflineSmall Offline
1
August 18, 2022 - 12:36 am
sp_Permalink sp_Print

I'm having trouble when merging two queries and expanding the result.

In the attached (see the 'All Assets & Revisions' query), in the third step when I expand the merged data, the values in the Revision Date and Monthly Finance columns in records two and three get switched.  See the screenshots below. 

I cannot work out why this would happen.  Can anyone help?  The file is attached.

Capture1.PNGImage Enlarger

Capture2.PNGImage Enlarger

sp_PlupAttachments Attachments
  • sp_PlupImage Capture2.PNG (10 KB)
  • sp_PlupImage Capture1.PNG (11 KB)
Avatar
Riny van Eekelen
Örnsköldsvik, Sweden
Moderator
Members


Trusted Members

Moderators

Power BI
Level 0
Forum Posts: 440
Member Since:
January 31, 2022
sp_UserOfflineSmall Offline
2
August 18, 2022 - 1:32 am
sp_Permalink sp_Print

Hi Mike,

Changed your staging query as I didn't really understand why you included a grouping step. The attached file seems to do what you want.

If not, come back here.

Riny

Avatar
Mike Magill

Active Member
Members
Level 0
Forum Posts: 3
Member Since:
August 18, 2022
sp_UserOfflineSmall Offline
3
August 18, 2022 - 2:00 am
sp_Permalink sp_Print

Hi,

Thanks for looking at this.  I need to add an index that starts at 1 for each revision per Asset ID.  That's why I used a grouping step rather than just adding the index to the main table.

I suspected that the grouping step is causing the issue but I have no idea why.

Avatar
Riny van Eekelen
Örnsköldsvik, Sweden
Moderator
Members


Trusted Members

Moderators

Power BI
Level 0
Forum Posts: 440
Member Since:
January 31, 2022
sp_UserOfflineSmall Offline
4
August 18, 2022 - 2:42 pm
sp_Permalink sp_Print sp_EditHistory

Hi Mike,

Understood! I added an assets to see what exactly is happening. The grouping isn't causing the problem. The sorting step in the staging query is the cause. Can't explain why it happens but PQ and haven't seen this happen with such small data sets. But somehow, PQ looses track of things upon sorting. Then you can force PQ to "fix" the table into memory by adding the Table.Buffer function. Did that and now it works.

Riny

 

Edit:

If you would rename the Date columns in the Asset List and the Lease Revisions to just "Date" (i.e. not "Original Date" and "Revision Date") you could use the following query to create the desired output in one go, without the staging query.

let
Source = Table.Combine({#"Asset List", #"Lease Revisions"}),
Sort = Table.Sort(Source,{{"Asset ID", Order.Ascending}, {"Date", Order.Ascending}}),
Merge = Table.NestedJoin(Sort, {"Asset ID"}, Rebates, {"Asset ID"}, "Rebates", JoinKind.LeftOuter),
Expand = Table.ExpandTableColumn(Merge, "Rebates", {"Rebate"}, {"Rebate"}),
GroupedRows = Table.Group(Expand, {"Rebate"}, {{"Grouped", each _, type table [Asset ID=nullable text, Date=nullable date, Monthly Finance=nullable number, Rebate=nullable number]}}),
AddRev = Table.AddColumn(GroupedRows, "Custom", each Table.AddIndexColumn([Grouped], "Index", 0, 1, Int64.Type)),
Combine = Table.Combine (AddRev[Custom])
in
Combine

Avatar
Mike Magill

Active Member
Members
Level 0
Forum Posts: 3
Member Since:
August 18, 2022
sp_UserOfflineSmall Offline
5
August 18, 2022 - 7:19 pm
sp_Permalink sp_Print

Thanks.

My actual workbook is far more complicated than the attached (which I just produced to show the error) so your second solution won't necessarily work.

Buffering the table just after the sort step works though.

 

Thanks for your help.

Avatar
Riny van Eekelen
Örnsköldsvik, Sweden
Moderator
Members


Trusted Members

Moderators

Power BI
Level 0
Forum Posts: 440
Member Since:
January 31, 2022
sp_UserOfflineSmall Offline
6
August 18, 2022 - 8:05 pm
sp_Permalink sp_Print

Glad it worked!

Avatar
Philip Treacy
Admin
Level 10
Forum Posts: 1512
Member Since:
October 5, 2010
sp_UserOfflineSmall Offline
7
August 22, 2022 - 11:25 am
sp_Permalink sp_Print

Hi Mike,

data in PQ isn't stored in a consistent state - that is, the view you see of the data and the way PQ is actually storing it, isn't necessarily the same.  As you've seen with the sorting, you are expecting the data to look one way, but you get a different result than expected because PQ is actually storing the data differently than you think.

You can force PQ to store the data in a known/consistent way by using Buffer functions like Table.Buffer and List.Buffer.  These Buffer functions load the data into memory and the data is then stored in an ordered way that is not changed until the data is released.

I've written a couple of blog posts that illustrate this behaviour and may help to shed light on the way PQ works

Dense Ranking in Power Query • My Online Training Hub

Quickly Create Running Totals in Power Query • My Online Training Hub

Cheers

Phil

sp_Feed
Go to top
Forum Timezone: Australia/Brisbane
Most Users Ever Online: 245
Currently Online: Brian Pham, drsven
Guest(s) 9
Currently Browsing this Page:
1 Guest(s)
Top Posters:
SunnyKow: 1432
Anders Sehlstedt: 870
Purfleet: 412
Frans Visser: 346
David_Ng: 306
lea cohen: 219
A.Maurizio: 202
Jessica Stewart: 202
Aye Mu: 201
jaryszek: 183
Newest Members:
drsven
Annie Witbrod
wahab tunde
Cong Le Duc
Faisal Bashir
Ivica Cvetkovski
Blaine Cox
Shankar Srinivasan
riyepa fdgf
Hannah Cave
Forum Stats:
Groups: 3
Forums: 24
Topics: 6205
Posts: 27211

 

Member Stats:
Guest Posters: 49
Members: 31880
Moderators: 3
Admins: 4
Administrators: Mynda Treacy, Philip Treacy, Catalin Bombea, FT
Moderators: MOTH Support, Velouria, Riny van Eekelen
© Simple:Press —sp_Information

Sidebar

Blog Categories

  • Excel
  • Excel Charts
  • Excel Dashboard
  • Excel Formulas
  • Excel PivotTables
  • Excel Shortcuts
  • Excel VBA
  • General Tips
  • Online Training
  • Outlook
  • Power Apps
  • Power Automate
  • Power BI
  • Power Pivot
  • Power Query
microsoft mvp logo
trustpilot excellent rating
Secured by Sucuri Badge
MyOnlineTrainingHub on YouTube Mynda Treacy on Linked In Mynda Treacy on Instagram Mynda Treacy on Twitter Mynda Treacy on Pinterest MyOnlineTrainingHub on Facebook
 

Company

  • About My Online Training Hub
  • Disclosure Statement
  • Frequently Asked Questions
  • Guarantee
  • Privacy Policy
  • Terms & Conditions
  • Testimonials
  • Become an Affiliate

Support

  • Contact
  • Forum
  • Helpdesk - For Technical Issues

Copyright © 2023 · My Online Training Hub · All Rights Reserved. Microsoft and the Microsoft Office logo are trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries. Product names, logos, brands, and other trademarks featured or referred to within this website are the property of their respective trademark holders.