Every material is different. Some materials, like diamond, are hard and others, like butter, are soft. This depends on what they are made of.
One way to think about a material is to imagine it made of tiny blocks of material held together by glue. If the glue is tough the whole material is hard, but if the glue is weak then the material will be soft.