Since its introduction in the Second Generation of the mainline Pokémon games, Shiny variants has become something of a status symbol. The culmination of a Pokémon Trainer trying to be the very best. Achieving something to show their dedication, passion and also luck. Many still haven’t found one, while others didn’t know and fleed from it. For many the very first encounter with a Shiny is the famous Red Gyarados at Lake of Rage in Gen II. Although some others could have found one as their own starter. Or infuriatingly the Shiny Poochyena‘s attacking Prof. Birch. Likewise the Professor Juniper’s Shiny Minccino at the introduction screen.
What is a Shiny
t’s a Pokémon with a different color than their regular form accompanied by a brief star-like animation at his entrance. A new variant called “Square Shiny” introduced in Gen VIII will show square animations, it’s generally rarer with a 1/ 16 chance. The color variation was added as a bonus from the developers to show off the colors of the Game Boy Color with Gen II. While the term has been adopted officially in Gen V, it was already used during events in Gen IV. In Gen II it was tied to the Pokémon DVs, resulting in higher than average stats. While from Gen III onward it was simply a visual difference.
As for the color scheme, it follows certain patterns, but there are definitely situation where the developers hand picked the Shiny colors. Generally for prominent Pokémon such as Legendaries, Mythicals or unique/iconic ones.
The odds
Up to Gen V the general odds were 1/8192, while it was later adjusted to 1/4096 from Gen VI onward. In Gen II the odds were related to certain combinations of Pokémon DVs (0-15). Interestingly enough, the same combinations were retroactively compatible with Gen I stats making it possible (but not visible) to obtain a Shiny This was later adjusted with Gen III new mechanics and stats (like Trainer ID which is secret, and IVs which replaced DVs).
Prerequisites & Methods
The odds of encountering one are definitely small but the developers added several mechanics to make them more accessible. Generally you’ll want to setup yourself to increase your odds as well as requiring less work to do if training them. The most important item to obtain is the Shiny Charm, a Key Item that will give 2 extra chances to find a Shiny. Locked behind Pokèdex completion, either National in older gen, to regional in newer one. Another useful thing is the use of Synchronize ability, to have a 50% of an encountered Shiny to have the same Nature as the user. This is done by positioning the Synchronizer in the first slot of the team (even fainted), and it’s useful in older gens where Nature cannot be changed. Also useful is the usage of a Pokémon with False Swipe and Paralysis moves, avoiding Pressure ability, to avoid self-knockout.
Soft reset
This is the most popular and older method to this date. Soft resetting is a combination of buttons which will reset the game without turning on/off the console. This will save a bit of time and make the method a bit more consistent. The general idea is to use this method for stationary Pokémon which are guaranteed encounters. Soft resetting until a shiny appears. As you may think this is indeed time consuming but also the least task intensive. This is still used to this date to obtain the non-shiny locked Legendaries, which can’t be bred.
Horde Battle
Introduced in Gen VI, it’s a feature that allows to battle 5 Pokémon at once. There’s a misconception that Horde Battles will allow 5/4096 odds for a Shiny. This is indeed false and each Pokémon will have the full odds, but will still allow to be a faster process than regular battles. This also have some downside, considering Multi-Hit moves and risks of knockouts from rival Pokèmon.
Breeding & Masuda
Introduced in Gen II and expanded with subsequent generations, this is the most popular method to this day. In Gen II you could obtain a Shiny by breeding one with a regular one to receive up to an odd of 1/64. It was due to being able to pass down parents DVs, in this case from the one carrying the Shiny “values”, onto the offspring. This was later removed in Gen III. But in Gen IV the method called Masuda, from the game director, was introduced thanks to the Nintendo Wi-Fi Connection. This method works as breeding Pokémon of different region/language will result in 5 (later 6) additional chances. Thus giving 1/1638(Gen IV), 1/1365(Gen V) and the current 1/683. Moreover thanks to the breeding mechanics it’s entirely possible to “craft” competitive Shinies.
Chaining
Introduced in Gen VI, this method was pretty popular back then and used the Pokéradar. It’s a lenghty method in which you had to take attention of bush shaking, while also choosing the best location to have the fewer bushes. The objective is to consecutively defeat in a chain the same species of Pokémon with increasing odds, capping at 40. This method requrie also a lot of attention, thinking and dedication. As generally the same Pokémon will appear in the farthest within 4×4 tiles from the player and will have the same shake animation. Failing to encounter the same species will result in the chain to reset. This was later reintroduced in Gen VI with the PokéNav, allowing to see which species is hiding in a grass patch and not breaking the chain if scaring a Pokèmon when sneaking.
Chain Fishing
A variant of Chaining introduced in Gen VI using the Fishing Road. By fishing in the same spot with each consecutive encounter (not necessary the same species) increasing the odds. It’s important to not break the chain by changing spot, rod or failing trigger the encounter. Thus abilities like Suction Cup or Sticky Hold, as well as external items that increase the odds of not failing are important.
Friend Safari
A method introduced in Gen VI. As part of the Post Game, it allows to have a 5/4096 to find a Shiny in the Friend Safari Zone. The more friends you have the more variety of Pokémon you’ll have access to.
SOS Battle
Introduced in Gen VII this method is a feature that allow wild Pokémon to call for help. This open the chance to obtain different and rare Pokémon, with Hidden Abilities, better IVs and Shiny. The chain max out at 255 and starting from 70 it also bump the odds to 1/1024. Each Pokémon has a chance to call for help when they are in a pinch with item and abilites to bump that odd. Some of them could also call “rival” that could knock the caller down, ending the chain. Thus it’s important to have a proper setup before starting one.
Encounter Boost
As part of the overall Gen VIII experience, there are extra odds to find a Shiny just by playing. By catching or defeating the same species up to 500, the Brilliant Pokèmon has up to 6x chances of being Shiny.
Dynamax Adventure
Introduced in the Gen VIII Crown Tundra DLC. It drastically improve the odds to 1/300 and it is rolled at the end of the Adventure. Moreover if the player has the 4 slots occupied by the 3 Pokémon+ the companion if will jump to 4/ 300, translating into a 1/75.
Sparkling Power Sandwich
A mechanic introduced in Gen IX with the picnic option. While setting up a camp it’s possible to crete special sandwiches with various effects. The Sparkling Power one allow extra odds of finding a Shiny, exactly up to 4 extra times. Thus a chance of 1/1024, needing Salty Herba Mystica along with other ingredients. As each variant of the recipe will attract different type of Pokémon.
Mass Outbreaks
Part of daily events in Gen IX, these outbreak comes with extra odds. By defeating 30 Pokémon while setting a camp will give 1/2048 chance. Maxing out at 1/1365 with 60 battles. This is generally combined with Sparkling Power Sandwich for even greater odds.
Cute Charm (Glitch)
A particular glitch discovered in Gen IV. Specific combinations of Trainer ID and Secret ID (which is hidden), bump the chances to an astronomical 66.67%! This activate if the leading Pokémon has the ability Cute Charm and any encounter is a species that have both gender. Thus genderless and 100% single gender won’t be affected.
Guaranteed
Although not many, Game Freak has decided to reward players with some guaranteed shinies. The most prominent is Lake of Rage’s Red Gyarados in the Gen II games and remakes. This Shiny Gyarados in Gen II was always male, thus also useful for Gen II Shiny odds via breeding.
Peculiar to Crystal is the Odd Egg, given by the Day Care Man. Containing one of the 7 Baby Pokémon introduced, having 14% chance to be Shiny. The chance is determined the moment you accept it, so just by saving before receiving and a few retries will net you a Shiny. 
Gen V sequels, as in Black&White 2, offered Shiny Gible or Dratini. Only those who manage to defeat Benga at the Black Tower or White Treehollow during the Postgame.
And again in those games, by completing/seeing the whole National Pokédex (bar Mythicals), you’ll have access to the Nature Preserve. There, a fully evolved Shiny Haxorus will wait to be captured. 
Gen VII‘s Ultrawormwhole, hides apart from non Shiny-locked Legendaries, also insane chances of Shiny for preset Pokémon. Starting from a minimum of 1% up to 36% with 5000+ Light Years for triple rings portals. 
Little mention are also the Shiny Pokémon via distribution events, mostly online nowadays. These will let you, for the most part, give you Shiny of Legendaries, Mythicals and other Shiny-locked pocket monsters.
My Story
My personal story with Shinies dates back in Gen II, Gold Version. Apart from the usual Red Gyarados, I found several times a Shiny Raticate in Route 39. But back then I didn’t knew much and never caught it. Another big fail of mine was on my first playthrough of Sapphire in Gen III. There I found a Shiny Electrike in Route 111, but I didn’t have enough Pokéballs and even a Great Ball failed at that. That was the turning point for me, I always made sure to have enough Pokéball and at least a Paralyzer in my team.
After that my very first caught Shiny was also a risky one. I started a new file in Sapphire and found a Shiny Oddish in the Safari Zone cathcing it in 2 balls. Risky because while Oddish has high capture rate, it could still flee. I still have it to this day, now evolved into a Bellossom. Funnily enough I managed to find 2 Shines 5 minutes apart in Fire Red. Strangely enough they were the Unown A and Z while I was exploring the Tanoby Chambers.
After that I pretty much stopped with the games, skipping them all until Gen VII. As a new gen not only I learned of the various changes but also discovered the new methods and odds. The first random encounter Shiny in Moon was a Trubbish, after almost 200 hours and a Shiny Charm. After that I actively used SOS and Masuda and obtained over 50 shinies, with a bunch of competitive viable. With Ultra Sun I also expanded even more and also obtained a Shiny Zapdos via Ultrawormhole, moreover caught in a Beast Ball, among many other. Obtained a few Gift Pokémon by Soft Resetting for Nature. As well as imported Shiny Legendaries from the Gen II Virtual Console. Including Lugia, Ho-Oh, Suicune and Shiny Celebi through the global release of Crystal’s once Japan exclusive ingame event.
Conclusion
The popularity of the Shiny concept, not only has translated into the TCG but also all over the internet. With plenty of content creators creating content around them. Moreover the Shiny term used by the community was finally accepted by Game Freak as the official one. You can check the current events and distribution regarding Shiny and more on the Pokémon Official page