In pattern recognition systems, when no labeled data is available, what are some common unsupervised learning algorithms for pattern recognition, that can be used?
There are some unsupervised learning algorithms that can be used for pattern recognition (i.e. the discovery of patterns in data). The most notable one is probably k-means, which is a clustering algorithm. In k-means, you cluster your unlabeled data into groups (or clusters) based on the distance (or similarity) between them. When a new data point arrives, you'll associate it with the most similar cluster. In this sense, you are performing pattern recognition in an unsupervised way.
Here's an excerpt from the famous book Pattern Recognition and Machine Learning (2006) by C. Bishop
In other pattern recognition problems, the training data consists of a set of input vectors $x$ without any corresponding target values. The goal in such unsupervised learning problems may be to discover groups of similar examples within the data, where it is called clustering, or to determine the distribution of data within the input space, known as density estimation, or to project the data from a high-dimensional space down to two or three dimensions for the purpose of visualization.
So, there are other problems, apart from the problem of clustering, that you may want to solve with unsupervised learning algorithms for the purpose of pattern recognition, such as density estimation (see e.g. mixture models) or dimensionality reduction (see e.g. PCA).