Talk:Markov chain

Convergence speed to the stationary distribution

I am studying the Markov chain now and I found some possible typo.

> π(k) approaches to π as k → ∞

I believe it should be "π(k) approaches a1*π as k → ∞".

> In other words, π = ui ← xPP...P = xPk as k → ∞.

It should be π = u1 instead of ui. Here, u1 represents the eigenvector of P corresponding to lambda1=1.

— Preceding unsigned comment added by Jgdyjb (talk • contribs) 05:02, 15 August 2023 (UTC)

Thanks for this. I believe you are correct and that I've fixed it here. In future, when you find a problem you need to fix it yourself as generally no-one else will. — Bilorv (talk) 09:33, 21 August 2023 (UTC)

This can be far more comprehensible for students.

Students may need to understand how to apply Markov Chain computations to the following example. A shepherd has a flock of sheep, three pastures and two sheep dogs, to graze seven days a week. He grazes the pastures on a rotating basis, working one dog at a time on alternating days. Say one is black and one is white. My student needs to compute when will the shepherd will work the black sheep dog in pasture 1 on a Thursday/all Thursday occurrences. I can make the chain more interesting by adding variables, and apply it to retail sales, cell phone tracking and re-stocking needs, for a mathematical application.

My interest in this discussion derives from an appreciation of exactly how good the IDEA encryption algorithm is, understanding that it derives from substitution on a password dependent Markov chain with an astronomical period/periodicity.

While it may be theoretically vulnerable, it is exceedingly difficult to cryptanalyze, and it should be understood by ENTRY LEVEL USERS to be very adequate, even for life and death uses. Formfollows (talk) 04:45, 3 October 2023 (UTC)

Awful article

Scientific articles are Wikipedia's Achilles heel, but this is truly terrible. I'm trying to understand section A non-Markov example and I'm giving up. That's just not the way to do it. The counter-example should be simple and clear, not so complicated and obscure. And if you do such complicated examples, you need to add drawings.

It is also questionable description in Applications. Statistics? It is all statistics. Economics and finance? Ok, but next Social sciences includes economics. Finance could be separated from economics and then social sciences combined with economics. Pawel.jamiolkowski (talk) 18:32, 9 June 2024 (UTC)

Hi @Pawel.jamiolkowski,

For your feedback to be useful, you have to be more specific. Is your problem with the stochastic process itself, or with the way it is presented? How do you think that a drawing could help here?

Regarding your comment on the section Applications: listing "statistics" is perfectly relevant; please have a look into the difference between probability theory and statistics to see why the study of Markov chains is not a subfield of statistics. I think grouping economics and finance is also relevant here, but I agree that something more specific than social science could be used (especially considering that the meaning of the word is a bit vague and has changed over time). However to me these are pretty minor issues.

Malparti (talk) 08:31, 10 June 2024 (UTC)

First of all, A non-Markov example is the sub-section of Examples, right? Reader reads examples which are brief description without details. And it's okay. And then we asking: Ok, so what process won't be Markov chain? The reply is below. However instead seeing a similarly short description, we see loooong piece of writing.

It should be consistent with the previous part. Short decription for intuitive understanding.

I don't know what drawings could be included here, because I don't understand this.

And secondly, why is a separate article with the same title? I guess to deepen the topic. There should actually be such a counter-example there (although better written anyway - first an intuitive description, then a deeper explanation). Pawel.jamiolkowski (talk) 13:49, 10 June 2024 (UTC)

Hi again @Pawel.jamiolkowski,

The subsection is actually rather short; what makes it look long is in part the explanation of how to build a Markov chain out of this non-Markovian process (which is somewhat irrelevant — here we want to understand what makes a process non-Markovian; not how to make non-Markovian processes Markovian by augmenting the state-space...). My guess is that this is a recent addition; articles on basic and widely used math tools have a natural tendency to grow because they attract a lot of readers from different backgrounds and many people want to add something — so they have to be trimmed down once in a while. I will probably remove the last paragraph of the subsection, but there are other problems I would like to fix first.

Another problem might be the use of the terms "dime" and "nickel" and "quarter", which lengthens the text and make is look more complex than it is to non-American readers. I guess that could be improved, too.

Other than that, I think this example is about as simple at it can get without getting something entirely trivial, and I think the level of detail is more-or-less OK. So I think it would be helpful if you could invest time to understand it and then explain what you found hard to understand, and how the description could be improved.

Best, Malparti (talk) 22:27, 10 June 2024 (UTC)

You have almost the same example, but simple and clear here: Markov property

Do you see? It should look like that. 3 balls, colors etc.

But generally for grasping idea, at the beginning it should just explain that the essence of difference lies in difference between draw with replacement and draw without it. It's obvious that if we have constant number of objects and draw without replacement, the future depends on draw, because each draw corresponds with each ball / coin, 1:1. As the number of objects decreases, it is obvious that the probability of guessing increases. Everyone can understand it without any problem. Pawel.jamiolkowski (talk) 23:47, 10 June 2024 (UTC)

I agree that this example has some advantages; but I disagree that it is strictly better, at least in the way it is presented. For instance, it is not explicitly stated what is the process of interest. This is not ideal, especially since if we were to model everything that is described verbally, we would of course get a Markovian process. But I agree that something like "the color of the n-th draw in a Polya urn process" seems like a more natural counter-example than the example described in the article. I will try to take care of this over the weekend. Malparti (talk) 13:32, 12 June 2024 (UTC)

Unnecessary citations

Just dropping in to say that a book being part of ChatGPT['s] training set [1] is the worst argument for citing a reference that I have seen in weeks.

Also, the opening line of the "Applications" section is a summary of the text that follows and doesn't need a citation itself. XOR'easter (talk) 20:47, 12 June 2024 (UTC)

@XOR'easter: Indeed... So it seems my concerns were not unfounded. ;). It also seems some people know how to use different IPs.

If that's OK with you I've moved your comment to a new section, to keep it separated from the discussion with Pawel.jamiolkowski.

Best, Malparti (talk) 23:05, 12 June 2024 (UTC)

Semi-protected edit request on 13 June 2024

HI,the page was bloked, however, some users abused the system and erased Gagniucs book with no reason. I wish to insert Gagniucs book again, and the main reason is the fact that is the most academically cited book listed on this page. More ... the suspicious reason for wich a top book was erased on no grounds at all except replacement with other unknown sources that have no academic grounds to be cited. 213.233.108.161 (talk) 20:40, 13 June 2024 (UTC)

Declined. There was no "abuse" of "the system". Gagniuc's book was removed for multiple reasons, any one of which would have been sufficient by itself. "Most academically cited book listed on this page" is a meaningless standard, particularly since the book could well have benefited from having been advertised here for so long. XOR'easter (talk) 20:51, 13 June 2024 (UTC)