On 7 September 1903 SC Rio Grande, the oldest existing football club in Brazil, played an exhibition contest in Porto Alegre. Businessman Cândido Dias went to watch the game, but the ball went empty during the game and he was the only one who had another ball. After the game he started talking to players and it was decided to set up a football club in Porto Alegre. On September 15, the players met in a restaurant and founded Grêmio Foot-Ball Porto Alegrense, most of the founders were English or German immigrants. On March 6, 1904, the first official game was played against Fussball Club Porto Alegre, which was formed on the same day as Grêmio, and was won 1-0. The club won the Troféu Wanderpreis, which can still be seen in the club museum. Five months later, the first stadium was opened with the Baixada. The Wanderpreis was played until 1912 between Grêmio and Fussball Club.

Football was not very popular in Brazil these days and there were not many clubs. On July 18, 1909, Grêmio defeated SC Internacional 10-0. Internacional, which later on would become a club with the same success as Grêmio, only existed for two weeks. Goalkeeper Kallfelz of Grêmio would have talked to the supporters during the game because he did not have much to do. In 1910 there was a city competition in Porto Alegre, but in the first season Grêmio had to leave this title to Militar, the next two seasons won the club. On July 7, 1911, the club played against the national team of Uruguay and won 2-1. In 1912 the club won 23-0 against Nacional, Sisson scored 14 times in this biggest win ever for the club. In 1913 the club left the league after three games because a referee would have treated Grêmio unfairly. Together with several other clubs, Grêmio founded the rival AFPA and became an authoritarian champion. After two seasons the unions went together again. In 1919 the club would only win a new title.

A year earlier Grêmio was one of the founders of the Fundação Rio-Grandense de Desportes, the later Federação Gaúcha de Futebol. This federation would organize the state championship from 1919, as a result of which the city championship lost its prestige. The competition was in contrast to other state championships until 1960 a final round, which preceded regional competitions. Grêmio and Brasil de Pelotas played the first final, which was won by Brasil with 1-5. A year later, Grêmio had to leave the title to Guarany de Bagé, but in 1921 it was finally price and followed the first of many state championships. Under the bar in the team was club legend Eurico Lara, who would play for the club until his sudden death in 1935. In 1931 Grêmio was one of the first clubs to install a lighting so that evening matches could be played. On May 19, 1935, Grêmio Santos defeated 3-2 to become the first team from the state to beat a club from the state of São Paulo, traditionally seen as the strongest state. In 1950 Grêmio was the first club from outside Rio de Janeiro to play in the famous Maracanã, the club won 1-3 of Flamengo

Grêmio also played international competitions. In 1932 the club played for the first time abroad, in Uruguay. In 1949 the Uruguayan Nacional was defeated 3-1, when they returned to Porto Alegre the players were welcomed as heroes. That same year the club played for the first time in Central America. In 1953-1954 the club traveled to Colombia, Ecuador and Mexico to play matches. On 25 February 1959, Grêmio defeated Boca Juniors with 1-4 and thus became the first foreign team to win in the Bombonera. In 1961 the club made a European tour and played 24 matches in eleven countries.

In the state championship, the club was outclassed by city rival Internacional and won only a sporadic title. This changed in 1956: from that year until 1968 the club won the title every year except for the 1961 season. Hereafter, Internaciola invariably won the title until 1976 and Grêmio had to settle for second place. From 1985 to 1990 they became champion six times in a row. Hereafter, the club won the title every few years. After the title of 2010, it was not until 2018 that the club could become state champion again.

In 1959 the Taça Brasil started, for the first time there was a big competition with the winners of the state championships. During the first three seasons, the club was quickly eliminated and in 1962 they did not participate. In 1963 the club reached the semi-finals for the first time and then lost it from Santos. Also in 1967 the semi-final was reached, now Palmeiras was too strong. At the last edition the club was killed in the group phase.

In 1967, a second national competition was set up with the Torneio Roberto Gomes Pedrosa, which, like the Taça, delivered a national champion. In the first edition, the participants were divided into two groups and Grêmio finished second behind Palmeiras and was allowed to go to the second group stage with four clubs, where they were last. Attacker Alcindo played in this time for the club, he is one of the top shooters of all time for the club. Also the following editions could not break the pot.

In 1971 the current national competition started. Grêmio player Néstor Scotta was the very first player to score a goal in this competition, in the game against São Paulo FC. In the seventies the club had varying success and finished twice in fifth place.

The golden period for the club were the eighties. In 1980, the club already finished in sixth place. In 1981 the club reached the final for the title, against São Paulo. At home, the club won 2-1 thanks to two goals from Paulo Isidoro. At the return match in the Estádio do Morumbi, Baltazar was able to score the only goal that earned Grêmio the title of champion. In the semi-finals, before the game against Ponte Preta, however, no top club, 98.421 spectators in the home stadium of Grêmio, turned out to be an absolute record. Grêmio even lost the game with 0-1, but had won 2-3 in Campinas. A year later, the club again reached the final, now against Flamengo. Tonho Gil gave the club the lead in the Maracanã in the 83rd minute, but Zico made it in the 89th minute. The return game ended on a scoreless draw and there was a third game. Flamengo player Nunes already scored in the tenth minute, making Flamengo the champion. In the Copa Libertadores of that year, the club was eliminated in the group stage. A year later, however, it was price there. Grêmio became group winner and reached the final against Peñarol via the second group stage. In Montevideo, Tita brought the score to 0-1 after fifteen minutes, but Morena was still able to equalize. In the return game, Caio and César led to a 2-1 victory that brought in the title and also allowed them to play the intercontinental cup against Hamburger SV. The game was played on December 11, 1983 in Tokyo. Renato Gaúcho scored twice, Hamburg only once. Goalkeeper Mazarópi and Uruguayan defender Hugo de León were the other heroes in the game. This made Grêmio the world champion for the first and so far only time. That year the club in the Série A finished only fourteenth. As title defender, the club was also allowed to participate in the Copa Libertadores in 1984 and reached the final again. However, the club now lost against Independiente.

In the following years, the results were again variable. In 1989 the club won the very first edition of the Copa do Brasil. In the semi-finals the Flamengo team humbled 6-1, the final was won by Sport. As a result, the club qualified for the Copa Libertadores, but was eliminated in the group stage. After a third place in 1990, Grêmio finished in the penultimate position the following year, which meant that they had to take a step back for the first time in their existence. That year the club played the final of the Copa again and lost to Criciúma, so they were allowed to play in the Copa CONMEBOL the following year. Here they were eliminated in the quarterfinals by the Ecuadorian club El Nacional. In the Série B, the club finished only in ninth place, but because the Série A was expanded to 32 teams in 1993, the club promoted. On the return, the club finished eleventh and again played the cup final, which they now lost to Cruzeiro.

In 1994, the club in the Copa CONMEBOL was eliminated in the eighth finals by São Paulo. The club also reached the final of the Copa do Brasil and was now able to win from Ceará thanks to an early goal by Nildo in the return game, after the first leg ended 0-0. The club was then allowed to take part in the Copa Libertadores back in 1995 and was second behind Palmeiras in the group stage. In the second round, the club met Paraguyan Olimpia and won easily. In the quarterfinals, the club again hit Palmeiras. Grêmio won 5-0 and the semi-final seemed a certainty, yet it was still shaky for the club, as it became 1-5 in the return game. After the club also put Emelec aside, Grêmio placed himself for the third time for the final against the Colombian Atlético Nacional. After an own goal from Atlético, Mário Jardel and Paulo Nunes could score the score, but it was still 3-1. In the return match, it ended immediately and the club won the trophy for the second time. In the intercontinental cup the club lost after penalty kicks of Ajax Amsterdam.

In 1996, the club also had another success in the Série A. The club finished sixth in the regular season, but then eliminated Palmeiras and Goiás on the way to the final. In it, the club found Portuguesa, which had become eighth in the competition. Portuguesa won at home with 2-0 and Grêmio had to win with the same score or better in the return match to become champion. Thanks to goals from Paulo Nunes and Ailton, it became 2-0 and Grêmio became national champion for the second time. Against Independiente, the club had already won the Recopa Sudamericana earlier that year and in the Copa Libertadores the club reached the semi-finals, which they lost to América de Cali. In 1997 the club won the cup for the third time and reached the quarter final against Cruzeiro in the Copa Libertadores. Also in 1998, a compatriot, now Vasco da Gama, eliminated the club at this stage of the tournament.

In 2001 the club won the cup for the fourth time, now in the final against Corinthians. In the Copa Mercosur that year, the club lost the semi-finals of Flamengo. In the subsequent Copa Libertadores, Grêmio reached the semi-final, which she lost to Olimpia. In the Série A, the club finished third. After this it went downhill. In the Copa Libertadores of 2003, the club still played the quarterfinals against Independiente Medellín, but in the league the club became the twentieth and the year at the 24th and last, causing the club to relegate for the second time in history.

The club immediately became champion in the Série B and so restricted the elite to one year. On the return, the club immediately finished in third place, with two points behind arch-rival Internacional. This was sufficient to qualify for the Copa Libertadores and barely a few years after the relegation the club had international success again. After the group stage, Grêmio defeated São Paulo, Defensor and Santos and stranded in the final against Boca Juniors. After a sixth place in the Série A of that year, the club qualified for the Copa Sudamericana 2008, where they immediately flew away against city rival Internacional. In the Série A, the club only had to let São Paulo go that year. In the Copa Libertadores the club only lost in the semifinals of Cruzeiro. Two years later, the club went out in the second round against Universidad Católica. In 2012 and 2013 the club became third and second respectively in the Série A, but in the subsequent Copa Libertadores invariably went in the second round. In 2017, Grêmio captured the Copa Libertadores by defeating the Argentinean CA Lanús in the final over two games.