• Skip to main content
  • Skip to header right navigation
  • Skip to site footer

My Online Training Hub

Learn Dashboards, Excel, Power BI, Power Query, Power Pivot

  • Courses
  • Pricing
    • Free Courses
    • Power BI Course
    • Excel Power Query Course
    • Power Pivot and DAX Course
    • Excel Dashboard Course
    • Excel PivotTable Course – Quick Start
    • Advanced Excel Formulas Course
    • Excel Expert Advanced Excel Training
    • Excel Tables Course
    • Excel, Word, Outlook
    • Financial Modelling Course
    • Excel PivotTable Course
    • Excel for Customer Service Professionals
    • Excel for Operations Management Course
    • Excel for Decision Making Under Uncertainty Course
    • Excel for Finance Course
    • Excel Analysis ToolPak Course
    • Multi-User Pricing
  • Resources
    • Free Downloads
    • Excel Functions Explained
    • Excel Formulas
    • Excel Add-ins
    • IF Function
      • Excel IF Statement Explained
      • Excel IF AND OR Functions
      • IF Formula Builder
    • Time & Dates in Excel
      • Excel Date & Time
      • Calculating Time in Excel
      • Excel Time Calculation Tricks
      • Excel Date and Time Formatting
    • Excel Keyboard Shortcuts
    • Excel Custom Number Format Guide
    • Pivot Tables Guide
    • VLOOKUP Guide
    • ALT Codes
    • Excel VBA & Macros
    • Excel User Forms
    • VBA String Functions
  • Members
    • Login
    • Password Reset
  • Blog
  • Excel Webinars
  • Excel Forum
    • Register as Forum Member

Duplicates|Power Query|Excel Forum|My Online Training Hub

You are here: Home / Duplicates|Power Query|Excel Forum|My Online Training Hub
Avatar
sp_LogInOut Log In sp_Registration Register
sp_Search Search
Advanced Search|Last Search Results
Search
Forum Scope




Match



Forum Options



Minimum search word length is 3 characters - maximum search word length is 84 characters
sp_Search Search
sp_RankInfo
Lost password?
sp_CrumbsHome HomeExcel ForumPower QueryDuplicates
sp_PrintTopic sp_TopicIcon
Duplicates
Avatar
Johan Vandeweijer
Member
Members
Level 0
Forum Posts: 15
Member Since:
March 3, 2021
sp_UserOfflineSmall Offline
1
June 30, 2021 - 7:42 pm
sp_Permalink sp_Print

Dear,

I now that it's very easy to remove duplicates via power query, but I have an example where I don't want to remove all duplicates. 

For example:

I have material where I have components with 2 cylinder ends which isn't correct. But I also have several disc's, which are correct.

But of the cylinder end, I need only the second, version 3 / M05803300 and I wanted to remove the version 4.

The example is showing only one material number, but I have more then 1.000 is my real example. And both components are existing otherwise it was easy. Can I remove duplicates, without touching all duplicates? See attachement "duplicates" for example

PlantMaterialMaterial DescriptionAlternative BOMExplosion levelComponentComponent DescriptionComponent Qty

STM1 M01167210 SHOCK ABSORBER 1 4 M05803000 CYLINDER END 1
STM1 M01167210 SHOCK ABSORBER 1 3 M05803300 CYLINDER END 1
STM1 M01167210 SHOCK ABSORBER 1 2 M10010192 DISC 1
STM1 M01167210 SHOCK ABSORBER 1 3 M100250A1 DISC 1
STM1 M01167210 SHOCK ABSORBER 1 3 M100250A3 DISC 1
STM1 M01167210 SHOCK ABSORBER 1 3 M100250A3 DISC 1
STM1 M01167210 SHOCK ABSORBER 1 2 M100252A7 DISC 1
STM1 M01167210 SHOCK ABSORBER 1 2 M100252F3 DISC 1
STM1 M01167210 SHOCK ABSORBER 1 2 M100252K2 DISC 1
STM1 M01167210 SHOCK ABSORBER 1 2 M100252K4 DISC 1
STM1 M01167210 SHOCK ABSORBER 1 2 M100252K5 DISC 1
STM1 M01167210 SHOCK ABSORBER 1 2 M10040108 DISC 1
STM1 M01167210 SHOCK ABSORBER 1 2 M10050301 DISC 1
sp_AnswersTopicSeeAnswer See Answer
Avatar
Mynda Treacy
Admin
Level 10
Forum Posts: 4447
Member Since:
July 16, 2010
sp_UserOfflineSmall Offline
2
July 1, 2021 - 10:00 am
sp_Permalink sp_Print

Hi Johan,

If you don't want the 4 Cylinder End then I'd create another column that concatenates the Explosion Level and Component columns together, then you can use the Filters on that new column to filter out the 4 Cylinder End records.

Mynda

Avatar
Johan Vandeweijer
Member
Members
Level 0
Forum Posts: 15
Member Since:
March 3, 2021
sp_UserOfflineSmall Offline
3
July 1, 2021 - 10:47 pm
sp_Permalink sp_Print

Hey Mynda,

Thanks for your reply.

In those examples, it was indeed explosion level 4, but it can be 3, 5  or 6 as well.

But is always be the highest level. I guess this is to complicated?

I added a few examples. 

Kind regards,

Johan

PlantMaterialMaterial DescriptionAlternative BOMExplosion levelComponentComponent DescriptionComponent Qty

STM1 M01167210 SHOCK ABSORBER 1 4 M05803000 CYLINDER END 1
STM1 M01167210 SHOCK ABSORBER 1 3 M05803300 CYLINDER END 1
STM1 M01167218 SHOCK ABSORBER 1 4 M05803000 CYLINDER END 1
STM1 M01167218 SHOCK ABSORBER 1 3 M05803300 CYLINDER END 1
STM1 ME1021202 SHOCK ABSORBER 1 4 M05803300 CYLINDER END 1
STM1 ME1021202 SHOCK ABSORBER 1 5 M05803000 CYLINDER END 1
STM1 MEP121402 OE PACKED 1 5 M05803300 CYLINDER END 1
STM1 MEP121402 OE PACKED 1 6 M05803000 CYLINDER END 1
Avatar
Catalin Bombea
Iasi, Romania
Admin
Level 10
Forum Posts: 1807
Member Since:
November 8, 2013
sp_UserOfflineSmall Offline
4
July 2, 2021 - 4:08 am
sp_Permalink sp_Print sp_EditHistory

Hi Johan,

If you want to keep some records based on component description, we can build a workaround.

For example, you can add an Index column and another column with this formula:

if Text.Contains([Component Description],"CYLINDER END") then [Component Description] else [Component Description] & Text.From([Index])

This new column will contain values like this:

CYLINDER END

CYLINDER END

DISC3

DISC4

DISC5

This way, Cylinder will be identified as duplicates, but DISC will have an additional unique index added in order to avoid removing them.

You can use this column, with other columns that together will create a unique key to remove duplicates.

It can be any other column, not necessarily the description:

if Text.Contains([Component Description],"CYLINDER END") then [Material] else [Material] & Text.From([Index])

Avatar
Johan Vandeweijer
Member
Members
Level 0
Forum Posts: 15
Member Since:
March 3, 2021
sp_UserOfflineSmall Offline
5
July 2, 2021 - 9:41 pm
sp_Permalink sp_Print

Hey Catalin, 

Wow, this is working perfectly. I had to add the material as well, but this is working perfectly.

"if Text.Contains ([Component Description],"CYLINDER END") then [Material]&[Component Description] else [Material]&[Component Description]&[Index]"

Thanks a lot.LaughSurprisedWink

sp_AnswersTopicAnswer
Answers Post
sp_Feed
Go to top
Forum Timezone: Australia/Brisbane
Most Users Ever Online: 245
Currently Online: Velouria, Jack Brett, Ivica Cvetkovski
Guest(s) 11
Currently Browsing this Page:
1 Guest(s)
Top Posters:
SunnyKow: 1432
Anders Sehlstedt: 870
Purfleet: 412
Frans Visser: 346
David_Ng: 306
lea cohen: 219
A.Maurizio: 202
Jessica Stewart: 202
Aye Mu: 201
jaryszek: 183
Newest Members:
Hayden Hao
Angela chen
Sean Moore
John Chisholm
vexokeb sdfg
John Jack
Malcolm Toy
Ray-Yu Yang
George Shihadeh
Naomi Rumble
Forum Stats:
Groups: 3
Forums: 24
Topics: 6211
Posts: 27239

 

Member Stats:
Guest Posters: 49
Members: 31891
Moderators: 3
Admins: 4
Administrators: Mynda Treacy, Philip Treacy, Catalin Bombea, FT
Moderators: MOTH Support, Velouria, Riny van Eekelen
© Simple:Press —sp_Information

Sidebar

Blog Categories

  • Excel
  • Excel Charts
  • Excel Dashboard
  • Excel Formulas
  • Excel PivotTables
  • Excel Shortcuts
  • Excel VBA
  • General Tips
  • Online Training
  • Outlook
  • Power Apps
  • Power Automate
  • Power BI
  • Power Pivot
  • Power Query
microsoft mvp logo
trustpilot excellent rating
Secured by Sucuri Badge
MyOnlineTrainingHub on YouTube Mynda Treacy on Linked In Mynda Treacy on Instagram Mynda Treacy on Twitter Mynda Treacy on Pinterest MyOnlineTrainingHub on Facebook
 

Company

  • About My Online Training Hub
  • Disclosure Statement
  • Frequently Asked Questions
  • Guarantee
  • Privacy Policy
  • Terms & Conditions
  • Testimonials
  • Become an Affiliate

Support

  • Contact
  • Forum
  • Helpdesk - For Technical Issues

Copyright © 2023 · My Online Training Hub · All Rights Reserved. Microsoft and the Microsoft Office logo are trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries. Product names, logos, brands, and other trademarks featured or referred to within this website are the property of their respective trademark holders.