Store By the Way Bakery Lexington Ave with kosher section is located on 1236 Lexington Ave in New York. This is a special place, the kind you remember, but can never find. By the Way Bakery is an old-fashioned bakery where everything is made by hand, from scratch, in small batches. And, by the way, all of their baked goods are gluten-free and dairy-free. They're also certified kosher. Most important, everything the bakery make is as delicious as – if not better than – most conventional baked goods.